Exploring The Metropolitan Art Collections with Hume #2

In our last Metropolitan Art Collections post we ingested and processed part of a dataset containing more than 470,000 artworks from The Metropolitan Museum of Art and created a knowledge graph using Hume, GraphAware’s insights engine.

This time, we will have a look at four use cases demonstrating how to get insights from the knowledge graph. We will start with Hume Visualisations to explore tag’s context; create Hume Actions to analyse the donors, and finally, use the Graph Data Science Library to suggest similar paintings.

Exploring tag’s context

We do not need complex queries to find interesting facts in the Art knowledge graph. A simple exploration within the Hume Visualisation can reveal the context of a topic the user is interested in. As an example, let’s head back in history and take a look at one of the characters popular in the middle ages - Saint Jerome.

1. Let’s search for ‘jerome’ in the visualisation search bar and add the Tag ‘Saint Jerome’ to the canvas.
2. We expand the node and bring up all the paintings with this character.
3. Next, we select all the paintings by clicking on a painting and pressing cmd + K.
4. And continue with expanding the Tags, Medium, Century and Credit nodes
5. Finally, we set the styles to relative sizing according to the nodes degree relative to the visualisation and get the following graph:

At first glance it’s apparent that Robert Lehman donated the majority of these paintings. We can also see that the prominent medium is tempera on wood, together with golden ground. Most of the paintings belong to the 15th century. By inspecting the related tags, we can deduce that Jerome frequently appears at nativity scene paintings. Quite strange if we consider he lived at the end of the third century, isn’t it?

Multiple authors cooperating on one artwork

This use case is inspired by Neo4j’s Summer of Nodes challenge. We would like to find all artworks with more than one author and learn more about them. To do so, we create a new Global Action with this cypher:

MATCH path = (a1:Artist)<-[:ARTIST]-(p:Painting)-[:ARTIST]->(a2:Artist)
WHERE id(a1) <> id(a2)
RETURN path

After running the action and expanding the Medium and Classification, it is apparent that the artworks are separated into two clusters. The first one are paintings, mostly oil on canvas or wood. The second group is more interesting - it consists of Miniatures, the authors used ivory, gold and enamel. If we inspect the artworks closer, we discover that the second group is made of Snuff Boxes. And it makes perfect sense that a snuff box is an item which multiple artists need to cooperate on.

If we look at the nationalities, we can see that most of the cooperating authors are of the same nationality. There are only a few exceptions among British and American authors.

Top Donors

With another Global Action we can find the people and organizations who donated most artworks to the museum. The cypher for this action would be:

MATCH (o)<-[c:CREDIT]-(p:Painting) WHERE c.type = 'gift' OR c.type = 'bequest'
WITH o, count(c) as count
RETURN o ORDER BY count DESC LIMIT 10

We set the styling, so that the nodes are scaled according to their absolute degree in the database. When we expand the paintings and the time-periods where they belong to, we can get a rough overview of each donor, especially when highlighting his neighborhood.

For example J.P. Morgan mostly donated paintings from the second half of the 18th century; there were a few from the 15th and 16th century, some of which were donated together with Charles Wrightsman.

Finding a similar painting

The very base for a recommendation engine is being able to determine how similar two objects are. We will try to find similarities between two paintings. We will use the Tag, Medium and Century (time period) classes connected to the Paintings, to create a feature vector that represents the artwork. We save the features vector as an attribute of each painting.

MATCH (feature)
WHERE "Tag" in labels(feature) OR "Medium" in labels(feature) OR "Century" in labels(feature)
WITH feature
ORDER BY id(feature)
MATCH (painting:Painting)
OPTIONAL MATCH (painting)-[relatedTag:TAGGED]->(feature)
OPTIONAL MATCH (painting)-[relatedMedium:MEDIUM]->(feature)
OPTIONAL MATCH (painting)-[:MEDIUM]-(:Medium)-[relatedMediumApplied:ACTIVE]->(feature)
OPTIONAL MATCH (painting)-[belongsToPeriod:BELONGS_TO]->(feature)
WITH painting, collect(CASE WHEN relatedTag IS null THEN 0 ELSE 1 END) as featuresTag,
collect(CASE WHEN ((relatedMedium IS NOT null) OR (relatedMediumApplied IS NOT null))
              THEN 0.3
            ELSE (CASE WHEN (belongsToPeriod IS NOT null)
                          THEN 0.2
                        ELSE (
                          CASE WHEN (relatedTag IS NOT null)
                                  THEN 1
                               ELSE 0
                          END)
                  END)
        END) as features
SET painting.features = features
RETURN painting

Next, we will utilize the GDS library from Neo4j, and use cosine similarity to compare the feature vectors and create a [SIMILAR_TO] relationship between two Paintings if they are at least a bit similar. We will persist the similarity as an attribute of the relationship.

MATCH (p:Painting)
 WITH {item:id(p), weights: p.features} AS paintingData
 WITH collect(paintingData) AS data
 CALL gds.alpha.similarity.cosine.stream({
   data: data,
   similarityCutoff: 0.3
 })
 YIELD item1, item2, count1, count2, similarity
 WITH gds.util.asNode(item1) AS from, gds.util.asNode(item2) AS to, similarity
 MERGE (from)-[:SIMILAR_TO{cosineSimilarity: similarity}]->(to)

Finally, we will create a LOCAL action of GRAPH return type which would allow us to find similar paintings. It finds all the paintings connected with [SIMILAR_TO] relationship, sorts them by the similarity and returns top 5. Hume adds the parameter $id as the id of the node that the user selected to fire the action.

MATCH (p:Painting)-[s:SIMILAR_TO]-(p2:Painting) WHERE id(p) = $id
WITH p, p2, s
ORDER BY s.cosineSimilarity DESC LIMIT 5
RETURN p, p2, s

When testing the action on Severin Roesen’s Still Life with Strawberries, the results are satisfying - the suggested paintings share tags, they are all Still Lifes with fruit, all from the same time period, all painted with oil. But most importantly, they do look similar.

Of course similarity based only on these features has its limitations. Producing convincing similarities across the whole dataset would require much more experimentation and tuning. The purpose of this exercise is only to demonstrate how easy it is to start with similarity recommendations using Hume.

Conclusion

We have explored some of the use cases on an art knowledge graph in Hume, ranging from basic node expansion to reveal the context in the graph visualisation, to simple similarity recommendations.

E-mail us at info@graphaware.com, or call any of our offices to learn more about how Hume can help your organisation.

Going Deeper?

Learn more about recommendations

[Video] Malt Aware: Discovering What to Drink with Neo4j: The ability to build simple but effective recommendations is underrated. In this talk Luanne Misquitta, VP of engineering, focuses on how to produce good starting-point recommendations for whisky using Cypher that are of higher quality than those we see at our favourite online stores.