Open Source Algorithms
Article Search
Last Updated: January 6th, 2025
Article recommendations are based on proximity to a user in our social graph database, which can be viewed at Account > View the Graph on Aemula.
While our graph database runs on an Aemula-controlled server for efficient querying, the entire database can be recreated from publicly-accessible data at any time.
The graph is created by querying the Aemula subgraph, which is a publicly queryable GraphQL-indexed subgraph of the activity generated from the Aemula Smart Contract.
For suggested improvements or requested changes, email community@aemula.com.
We are in process of setting up the community governance of the open source algorithms, allowing community members to make pull requests of the source code from Github and propose changes for vote by the community.
Function explained with comments throughout:
// function to perform a simple search for article recommendations
// the user passes their account address to the function
export const simpleArticleRec = async (account) => {
// create a session with our graph database
const session = driver.session();
// force account address to lowercase for normalization
const lowerAddress = account.toLowerCase();
// variable to set the maximum number of articles we want to find
// this will increase as more articles are published
const MAX_ARTICLES = 10;
// create a set for tracking the articles we find in our search
// a set is used to efficiently check for duplicate articles
const articleSet = new Set();
try {
// search for users 2 connections away from our target user
// essentially, users with mutual connections to an article
// user -(supports)-> [some article] <-(also supports)- another user
// the query to our database (cypher for Neo4j):
// start the search at our user (id: $accountID)
// look for articles the user has supported or authored
// then look for users that supported or authored those articles
// make sure we didn't just loop back to our user we started from
// make sure there are no duplicate users
// call these users the "second layer"
const secondLayer = await session.run(
`MATCH (u:User {id: $accountId})-[:SUPPORT|AUTHOR]->(a1:Article {inCirculation: true})
MATCH (a1)-[:SUPPORT|AUTHOR]->(u2:User)
WHERE u2.id <> $accountId
RETURN DISTINCT u2 as secondLayerUser`,
{ accountId: lowerAddress }
);
// keep track of all these "second layer" users
const secondLayerUsers = secondLayer.records.map((rec) => rec.get('secondLayerUser'));
// search for new articles our second layer users support or author
for (const u2 of secondLayerUsers) {
//for all of the second layer users
// as long as we haven't already found all the articles we need
if (articleSet.size >= MAX_ARTICLES) break;
// get the second layer users account address
const u2Id = u2.properties.id;
// start another search query in our graph database from this user
// this query:
// starts at the second layer user
// checks if they have authored any articles
// if they have, see if our main user has read it
// if it hasn't been read, add it to the article set
// continue this check until we either:
// find 3 articles or
// run out of authored articles to look for
const authoredRes = await session.run(
`MATCH (u2:User {id: $u2Id})-[:AUTHOR]->(a2:Article {inCirculation: true})
WHERE NOT ( (u:User {id: $accountId})-[]->(a2) )
RETURN DISTINCT a2.id as articleId
LIMIT 3`,
{
u2Id: u2Id,
accountId: lowerAddress,
}
);
// add any articles we found in the above query to our set
for (const rec of authoredRes.records) {
// keep track of the articles IPFS CID
const articleCid = rec.get('articleId');
// make sure we haven't already added this article to our set
if (!articleSet.has(articleCid)) {
articleSet.add(articleCid);
// if we have hit our target # of articles, we can stop looking
if (articleSet.size >= MAX_ARTICLES) break;
}
};
};
// if we don't have enough articles yet, continue looking
// this time, we will check for articles supported by second layer users
if (articleSet.size < MAX_ARTICLES) {
// similar to above, step through each second layer user
for (const u2 of secondLayerUsers) {
// make sure we haven't already found all the articles we need
if (articleSet.size >= MAX_ARTICLES) break;
// get the second layer users account address
const u2Id = u2.properties.id;
// start another query:
// starting at the second layer user
// look for all articles they have supported
// make sure our main user hasn't already read the article
// if not, add it to the list of articles we've found
const supportedRes = await session.run(
`MATCH (u2:User {id: $u2Id})-[:SUPPORT]->(a2:Article {inCirculation: true})
WHERE NOT ( (u:User {id: $accountId})-[]->(a2) )
RETURN DISTINCT a2.id as articleId`,
{
u2Id: u2Id,
accountId: lowerAddress,
}
);
// for all the articles we just found
for (const rec of supportedRes.records) {
// pull their IPFS CID
const articleCid = rec.get('articleId');
// make sure we don't already have the article in our set
if (!articleSet.has(articleCid)) {
// if not, add it to the set
articleSet.add(articleCid);
// check if we have found all the articles we need
if (articleSet.size >= MAX_ARTICLES) break;
}
};
};
};
// check if we have found enough articles yet
// with more scale, we can add another check for neutral articles
// if we don't have enough articles, just randomly fill in the rest
if (articleSet.size < MAX_ARTICLES) {
// how many more articles do we need to find?
const remainingArticles = MAX_ARTICLES - articleSet.size;
// randomly pull articles from our database
// check if our main user has read them
// if not, add them to the list of articles we've found
const randomRes = await session.run(
`MATCH (u:User {id: $accountId}), (a:Article {inCirculation: true})
WHERE NOT (u)-[]->(a)
WITH a
ORDER BY rand()
LIMIT toInteger($remainingArticles)
RETURN a.id AS articleId`,
{
accountId: lowerAddress,
remainingArticles: remainingArticles,
}
);
// make sure we don't already have these articles in our set
for (const rec of randomRes.records) {
// pull the IPFS CID for the article
const articleCid = rec.get('articleId');
// make sure we don't already have it
if (!articleSet.has(articleCid)) {
// if not, add it to our set
articleSet.add(articleCid);
};
};
};
// we now have our set of articles we want to recommend to the user
// convert our set to an array
// technical note: this loses ordering
// not an issue since we rank articles with another function
return Array.from(articleSet);
} catch (err) {
// check in case any errors occur
console.error('Error fetching simple article recommendations:', err);
throw err;
} finally {
// when we are done, end our session with the database
await session.close();
}
};