In our last Metropolitan Art Collections post, we ingested and processed part of a dataset containing more than 470,000 artworks from The Metropolitan Museum of Art and created a knowledge graph using GraphAware Hume, GraphAware’s insights engine.
This time, we will have a look at four use cases demonstrating how to get insights from the knowledge graph. We will start with GraphAware Hume Visualisations to explore tags’ context; create Hume Actions to analyse the donors, and finally, use the Graph Data Science Library to suggest similar paintings.
Exploring the tag’s context
We do not need complex queries to find interesting facts in the Art knowledge graph. A simple exploration within the GraphAware Hume Visualisation can reveal the context of a topic the user is interested in. As an example, let’s head back in history and take a look at one of the characters popular in the Middle Ages – Saint Jerome.
- Let’s search for ‘Jerome’ in the visualisation search bar and add the Tag ‘Saint Jerome’ to the canvas.
- We expand the node and bring up all the paintings with this character.
- Next, we select all the paintings by clicking on a painting and pressing
cmd + K. - And continue with expanding the Tags, Medium, Century and Credit nodes
- Finally, we set the styles to relative sizing according to the node’s degree relative to the visualisation and get the following graph:
At first glance, it’s apparent that Robert Lehman donated the majority of these paintings. We can also see that the prominent medium is tempera on wood, together with golden ground. Most of the paintings belong to the 15th century. By inspecting the related tags, we can deduce that Jerome frequently appears in nativity scene paintings. Quite strange if we consider he lived at the end of the third century, isn’t it?
Multiple authors cooperating on one artwork
This use case is inspired by Neo4j’s Summer of Nodes challenge. We would like to find all artworks with more than one author and learn more about them. To do so, we create a new Global Action with this cypher:
MATCH path = (a1:Artist)<-[:ARTIST]-(p:Painting)-[:ARTIST]->(a2:Artist)
WHERE id(a1) <> id(a2)
RETURN path
After running the action and expanding the Medium and Classification, it is apparent that the artworks are separated into two clusters. The first one is paintings, mostly oil on canvas or wood. The second group is more interesting – it consists of Miniatures, the authors used ivory, gold and enamel. If we inspect the artworks more closely, we discover that the second group is made of Snuff Boxes. And it makes perfect sense that a snuff box is an item on which multiple artists needed to cooperate.
If we look at the nationalities, we can see that most of the cooperating authors are of the same nationality. There are only a few exceptions among British and American authors.
Top donors
With another Global Action, we can find the people and organisations that donated the most artworks to the museum. The cypher for this action would be:
MATCH (o)<-[c:CREDIT]-(p:Painting) WHERE c.type = 'gift' OR c.type = 'bequest'
WITH o, count(c) as count
RETURN o ORDER BY count DESC LIMIT 10
We set the styling so that the nodes are scaled according to their absolute degree in the database. When we expand the paintings and the time periods to which they belong, we can get a rough overview of each donor, especially when highlighting their neighbourhood.
For example, J.P. Morgan mostly donated paintings from the second half of the 18th century to Charles Wrightsman.
Finding a similar painting
The very basis of a recommendation engine is being able to determine how similar two objects are. We will try to find similarities between the two paintings. We will use the Tag, Medium and Century (time period) classes connected to the Paintings to create a feature vector that represents the artwork. We save the features vector as an attribute of each painting.
MATCH (feature)
WHERE "Tag" in labels(feature) OR "Medium" in labels(feature) OR "Century" in labels(feature)
WITH feature
ORDER BY id(feature)
MATCH (painting:Painting)
OPTIONAL MATCH (painting)-[relatedTag:TAGGED]->(feature)
OPTIONAL MATCH (painting)-[relatedMedium:MEDIUM]->(feature)
OPTIONAL MATCH (painting)-[:MEDIUM]-(:Medium)-[relatedMediumApplied:ACTIVE]->(feature)
OPTIONAL MATCH (painting)-[belongsToPeriod:BELONGS_TO]->(feature)
WITH painting, collect(CASE WHEN relatedTag IS null THEN 0 ELSE 1 END) as featuresTag,
collect(CASE WHEN ((relatedMedium IS NOT null) OR (relatedMediumApplied IS NOT null))
THEN 0.3
ELSE (CASE WHEN (belongsToPeriod IS NOT null)
THEN 0.2
ELSE (
CASE WHEN (relatedTag IS NOT null)
THEN 1
ELSE 0
END)
END)
END) as features
SET painting.features = features
RETURN painting
Next, we will utilise the GDS library from Neo4j, and use cosine similarity to compare the feature vectors and create a [SIMILAR_TO] relationship between two Paintings if they are at least a bit similar. We will persist the similarity as an attribute of the relationship.
MATCH (p:Painting)
WITH {item:id(p), weights: p.features} AS paintingData
WITH collect(paintingData) AS data
CALL gds.alpha.similarity.cosine.stream({
data: data,
similarityCutoff: 0.3
})
YIELD item1, item2, count1, count2, similarity
WITH gds.util.asNode(item1) AS from, gds.util.asNode(item2) AS to, similarity
MERGE (from)-[:SIMILAR_TO{cosineSimilarity: similarity}]->(to)
Finally, we will create a LOCAL action of GRAPH return type, which would allow us to find similar paintings. It finds all the paintings connected with the [SIMILAR_TO] relationship, sorts them by similarity and returns the top 5. GraphAware Hume adds the parameter $id as the id of the node that the user selected to fire the action.
MATCH (p:Painting)-[s:SIMILAR_TO]-(p2:Painting) WHERE id(p) = $id
WITH p, p2, s
ORDER BY s.cosineSimilarity DESC LIMIT 5
RETURN p, p2, s
When testing the action on Severin Roesen’s Still Life with Strawberries, the results are satisfying – the suggested paintings share tags, they are all Still Lifes with fruit, all from the same time period, all painted with oil. But most importantly, they do look similar.
Of course, similarity based only on these features has its limitations. Producing convincing similarities across the whole dataset would require much more experimentation and tuning. The purpose of this exercise is only to demonstrate how easy it is to start with similarity recommendations using GraphAware Hume.
Conclusion
We have explored some of the use cases on an art knowledge graph in GraphAware Hume, ranging from basic node expansion to reveal the context in the graph visualisation, to simple similarity recommendations.
E-mail us at info@graphaware.com, or schedule a live demo to learn more about how GraphAware Hume can help your organisation.
Going deeper?
Learn more about recommendations
[Video] Malt Aware: Discovering What to Drink with Neo4j: The ability to build simple but effective recommendations is underrated. In this talk, Luanne Misquitta, VP of engineering, focuses on how to produce good starting-point recommendations for whisky using Cypher that are of higher quality than those we see at our favourite online stores.
