Code Insights Pings

Code Insights pings allow us to quantitatively measure the usage and success of Code Insights. This page is a source of truth for detailed explanations, statuses, and implementations of our pings.

We keep this docs page up to date because pings are a vital component of our product knowledge and prioritization process, and a broken or incorrect ping impacts 3-5 months of data (because that's how long a fix takes to propagate).

Terminology

  • FE event - log events that we send by calling standard telemetry service on the frontend. These pings live only in the event_logs table. These typically represent user actions, such as hovers.
  • BE capture - pings that our BE sends to the ping store by checking/selecting data from database tables. Our backend periodically sends these pings to the event_logs table. These typically represent absolute counts across the entire instance.

Metrics

Additions count, edits count, and removals count

Type: FE event

Intended purpose: To track how many times customers have created, edited, and removed insights, by week.

Functional implementation: We track insight creating/editing/deleting events in the creation UI form and insight context menu component with standard telemetry service calls.

Other considerations: N/A

Hovers count

Type: FE event

Intended purpose: To track how many times users hover over a datapoint to see the tooltip on the graph, or "dig in" to the information.

Functional implementation: This ping works by firing an event on the client when a user hovers over a datapoint on a code insight.

Other considerations: N/A

UI customizations count

Type: FE event

Intended purpose: To track how many times users resize the insight graphs.

Functional implementation: This ping works by firing an event on the client when a user resizes a Code Insights graph on the page.

Other considerations: N/A

Data point clicks count

Type: FE event

Intended purpose: To track how many times users click a datapoint to get to a diff search.

Functional implementation: This ping works by firing an event on the client when a user clicks an individual data point of an insight graph, which takes them to a diff search.

Other considerations: Because this functionality does not yet exist for backend insights, it only tracks clicks on frontend insights.

Page views count

Type: FE event

Intended purpose: To track how many times users view insights pages.

Functional implementation: This ping works by firing an event on the client when a user views any /insights page, whether it's creating or viewing insights.

Other considerations: As we add new insights pages it's important to make sure we're adding pages to this counter.

  • Aggregation: By week
  • Event Code: InsightsPageView
  • PRs: #17805
  • Version Added: 3.25
  • Version(s) broken: 3.25-3.26 (not weekly)(fix PR), 3.30 (broken when switching to dashboard pages, didn't track dashboard views)(fix PR)

Unique page views count

Type: FE event

Intended purpose: To track how many unique users are viewing insights pages each week.

Functional implementation: This ping works by firing an event on the client when a unique user views any /insights page for the first time that week, whether it's creating or viewing insights.

Other considerations: As we add new insights pages it's important to make sure we're adding pages to this counter.

  • Aggregation: By week
  • Event Code: InsightsUniquePageView
  • PRs: #17805
  • Version Added: 3.25
  • Version(s) broken: 3.25-3.26 (not weekly)(fix PR), 3.30 (broken when switching to dashboard pages, didn't track dashboard views)(fix PR)

Org-visible insights count (Total)

Type: BE capture

Intended purpose: To track how many insights are visible by more than just the creator of the insight.

Functional implementation: we gather this on the backend by joining the insight_view and insight_view_grants tables and counting the insights with org level grants.

Other considerations: N/A

  • Aggregation: total time, by insight type
  • Event Code: InsightOrgVisible
  • PRs: #21671
  • Version Added: 3.29
  • Version(s) broken: 3.31-3.35.0 (doesn't handle backend insights) fix PR

First time insight creators count

Type: FE event

Intended purpose: To track the week and count of the first time a user(s) creates a code insight, of any type, on an instance. The sum of first time insight creators count over all time is equal to the total number of unique creators who have made an insight.

Functional implementation: This metric queries the insight table for new addition events, then filters by unique IDs that appeared for the first time that week.

Other considerations: TODO does this ping include creators who create via the API?

  • Aggregation: By week
  • Event Code: WeeklyFirstTimeInsightCreators
  • Version Added: 3.25
  • Version(s) broken: 3.31-3.35.0 (doesn't handle backend insights, other bugs)

Total count of insights grouped by step size (days)

Type: BE capture

Intended purpose: To track the x-axis (time window) set by users on frontend insights, to help prioritize features related to setting time windows.

Functional implementation: this metric runs on the backend over all the insights.

Other considerations: N/A

Code Insights View/Click Creation Funnels

Type: FE event

Intended purpose: These pings allow us to both understand how the view/click/view/click conversion funnel works for the creation flows of all existing types of insights, as well as smell-check other pings. The reason we use both "view" and "button clicks" in this funnel is that it's possible to view a page without "funneling through" via the prior page's CTA (for example: you can reach the creation/edit screen by the "edit" button, which does not involve logging a click on the "create search insight" button).

Functional implementation: These events fire on the frontend when the user takes the below actions.

Other considerations:

View series counts

Type: BE capture

Intended purpose: To track the number of view series, grouped by presentation type and generation method. Note: a "view series" differs from a "series" by being attached to a particular insight. A series can be attached to more than one insight.

Functional implementation: This is calculated by joining the insight_series, insight_view_series, and insight_view tables.

Other considerations: N/A

Series counts

Type: BE capture

Intended purpose: To track the number of series, grouped by generation method.

Functional implementation: This is calculated using the insight_series table.

Other considerations: N/A

  • Aggregation: total
  • Event Code: SeriesCounts
  • Version added: 3.34

View counts

Type: BE capture

Intended purpose: To track the number of insight views, grouped by presentation type.

Functional implementation: This is calculated using the insight_view table.

Other considerations: N/A

  • Aggregation: total
  • Event Code: ViewCounts
  • Version added: 3.34

Total orgs with dashboards

Type: BE capture

Intended purpose: To track the number of orgs with at least one dashboard.

Functional implementation: This is calculated using the dashboard_grants table.

Other considerations: N/A

Total dashboard count

Type: BE capture

Intended purpose: To track the total number of dashboards.

Functional implementation: This is calculated using the dashboard table.

Other considerations: N/A

Insights per dashboard

Type: BE capture

Intended purpose: To track statistics (average, min, max, median, std dev,) about how many insights are on each dashboard.

Functional implementation: These are calculated using the dashboard_insight_view table.

Other considerations: N/A