Automatic Discovery of Application Topology

Asserts Entity Graph is a four dimensional map of application topology, it contains service to service (x, y), infrastructure stack (z) and time (t). Therefore at any point in time it knows what was talking to what and where it was running. The topology map is continuously and automatically inferred by analysing metric and span labels or tags.

Asserts Entity Graph is stored in a graph database, for additional background information a previous blog post covered the concept in greater detail.

Graph Database

Before looking at graph databases, consider a simple example using a relational database.

A relational database to store students and their membership of various clubs could be modelled like this. One table holds the details for the students and another table holds the details of the various clubs. Any student can be a member of zero or more clubs and any club can have zero or more members. To manage the relationship information a third table is required. To find the clubs that Tom is a member of requires the use of all three tables.

SELECT s.name, c.name FROM students s, clubs c, student_club_xref x
WHERE s.name = "Tom" AND s.id = x.student_id AND c.id = x.club_id

A graph database uses nodes and edges to store information instead of tables that a relational database uses. A node is used to represent a thing, like a student or a club. The node has properties similar to columns in a table. Edges contain the relationship information and can also have properties.

Relational databases use Structured Query Language (SQL) to manage the data, graph databases have their own language (Cypher) which leverages the different data structure. To find out which clubs Tom is a member of the query looks quite different to SQL.

MATCH (s.Student)-[:member]->(c:Club) WHERE s.firstName = 'Tom'
RETURN s.firstName, c.name

Graph databases do not have a fixed schema like relational databases making it easy to add other relationships, for example Tom could be both a member and treasurer of the Chess Club.

Asserts Entity Graph

Prometheus metrics and Open Telemetry tracing spans have labels/tags, for example:

kube_pod_container_info{container="catalogue", namespace="robot-shop", container_id="docker://53d7e10b32b4f5bb3dd441a5d9ecb1291c", …}

"spanID": "134964d20f120f30"
"tags": [
	{
		"key": "container.id",
		"type": "string",
		"value": "53d7e10b32b4f5bb3dd441a5d9ecb1291c"
	}
]

Tracing is not mandatory to infer relationship information, for example kube state metrics provides a lot of the labels used to build service, pod, container, namespace relationships. Service to service relationships can be extracted from service mesh (Istio, Linkerd) metric labels and Asserts has an eBPF probe that may optionally be deployed. When tracing is available, the relationship data can be inferred from the span tags.

Asserts analyses the labels/tags as the data arrives, infers the relationships between the various application components and stores the results in a graph database.

The Entity Graph is more than just service to service mapping that is available from tracing, it also has depth representing the stack where the service runs. Finally it also tracks changes over time, such that the full topology is available for any historical point in time. The Asserts Entity Graph is a four dimensional model consisting of: service to service (x & y), the stack (z) and time (t).

Enhancement

The relationship rules may optionally be enhanced. As an example if node metrics with labels for data centre and rack are available.

up{datacenter="dc1", rack="A7", ip_addr="84.48.16.12", ...}

Using the UI or via code additional entities for data centre and rack are created then a relationship of CONTAINS is added for the node, linking it to the rack and the rack to the data centre. Now it's possible, for example, to find out in which rack a node is located or a list of all racks in a data centre.

Automatic Collation

The Entity Graph gives Asserts a deep understanding of the application environment that it is monitoring. Engineering teams can use Asserts to easily inspect application topology with simple queries such as catalogue connected services or following links in the Entities Explorer.

The power of the Entity Graph is used when an incident occurs. For example, when a response time SLO has accelerated burn rate, Asserts traverses the Entity Graph and looks for all related application components and checks if they have any Assertions firing in the same time window. Matching entities are collected on to a dynamically created dashboard, putting all the information engineering teams need for their analysis in one place. Eliminating the need to have 20 browser tabs open in a futile attempt at manual correlation. The team working on an incident can be much smaller because there is no longer a requirement to include members from other teams for their domain specific knowledge.

Install Asserts today for free, forever to gain total insight into your application stack and speed up incident resolution without calling in all your engineers.