GitLab

Site admins can sync Git repositories hosted on GitLab (GitLab.com and GitLab CE/EE) with Sourcegraph so that users can search and navigate the repositories.

To connect GitLab to Sourcegraph:

  1. Depending on whether you are a site admin or user:
    1. Site admin: Go to Site admin > Manage repositories > Add repositories
    2. User: Go to Settings > Manage repositories.
  2. Select GitLab.
  3. Configure the connection to GitLab using the action buttons above the text field, and additional fields can be added using Cmd/Ctrl+Space for auto-completion. See the configuration documentation below.
  4. Press Add repositories.

NOTE That adding code hosts as a user is currently in private beta.

Supported versions

  • GitLab.com
  • GitLab CE/EE (v10.0 and newer)

Repository syncing

There are three fields for configuring which projects are mirrored/synchronized:

  • projects
    A list of projects in {"name": "group/name"} or {"id": id} format.
  • projectQuery
    A list of strings with one pre-defined option (none), and/or an URL path and query that targets a GitLab API endpoint returning a list of projects.
  • exclude
    A list of projects to exclude which takes precedence over the projects, and projectQuery fields. It has the same format as projects.

Troubleshooting

You can test your access token's permissions by running a cURL command against the GitLab API. This is the same API and the same project list used by Sourcegraph.

Replace $ACCESS_TOKEN with the access token you are providing to Sourcegraph, and $GITLAB_HOSTNAME with your GitLab hostname:

curl -H 'Private-Token: $ACCESS_TOKEN' -XGET 'https://$GITLAB_HOSTNAME/api/v4/projects'

Repository permissions

By default, all Sourcegraph users can view all repositories. To configure Sourcegraph to use GitLab's per-user repository permissions, see "Repository permissions".

User authentication

To configure GitLab as an authentication provider (which will enable sign-in via GitLab), see the authentication documentation.

Internal rate limits

Internal rate limiting can be configured to limit the rate at which requests are made from Sourcegraph to GitLab.

If enabled, the default rate is set at 36,000 per hour (10 per second) which can be configured via the requestsPerHour field (see below). If rate limiting is configured more than once for the same code host instance, the most restrictive limit will be used.

NOTE Internal rate limiting is only currently applied when synchronising campaign changesets.

Configuration

admin/external_service/gitlab.schema.json

{
	// If non-null, enforces GitLab repository permissions. This requires that there be an item in the `auth.providers` field of type "gitlab" with the same `url` field as specified in this `GitLabConnection`.
	"authorization": null,

	// TLS certificate of the GitLab instance. This is only necessary if the certificate is self-signed or signed by an internal CA. To get the certificate run `openssl s_client -connect HOST:443 -showcerts < /dev/null 2> /dev/null | openssl x509 -outform PEM`. To escape the value into a JSON string, you may want to use a tool like https://json-escape-text.now.sh.
	"certificate": null,
	// Other example values:
	// - "-----BEGIN CERTIFICATE-----\n..."

	// When set to true, this external service will be chosen as our 'Global' GitLab service. Only valid on Sourcegraph.com. Only one service can have this flag set.
	"cloudGlobal": false,

	// A list of projects to never mirror from this GitLab instance. Takes precedence over "projects" and "projectQuery" configuration. Supports excluding by name ({"name": "group/name"}) or by ID ({"id": 42}).
	"exclude": null,
	// Other example values:
	// - [
	//     {
	//       "name": "group/name"
	//     },
	//     {
	//       "id": 42
	//     }
	//   ]
	// - [
	//     {
	//       "name": "gitlab-org/gitlab-ee"
	//     },
	//     {
	//       "name": "gitlab-com/www-gitlab-com"
	//     }
	//   ]

	// The type of Git URLs to use for cloning and fetching Git repositories on this GitLab instance.
	//
	// If "http", Sourcegraph will access GitLab repositories using Git URLs of the form http(s)://gitlab.example.com/myteam/myproject.git (using https: if the GitLab instance uses HTTPS).
	//
	// If "ssh", Sourcegraph will access GitLab repositories using Git URLs of the form [email protected]:myteam/myproject.git. See the documentation for how to provide SSH private keys and known_hosts: https://docs.sourcegraph.com/admin/repo/auth#repositories-that-need-http-s-or-ssh-authentication.
	"gitURLType": "http",

	// Defines whether repositories from this GitLab instance should be enabled and cloned when they are first seen by Sourcegraph. If false, the site admin must explicitly enable GitLab repositories (in the site admin area) to clone them and make them searchable on Sourcegraph. If true, they will be enabled and cloned immediately (subject to rate limiting by GitLab); site admins can still disable them explicitly, and they'll remain disabled.
	"initialRepositoryEnablement": null,

	// An array of transformations will apply to the repository name. Currently, only regex replacement is supported. All transformations happen after "repositoryPathPattern" is processed.
	"nameTransformations": null,
	// Other example values:
	// - [
	//     {
	//       "regex": "\\.d/",
	//       "replacement": "/"
	//     },
	//     {
	//       "regex": "-git$",
	//       "replacement": ""
	//     }
	//   ]

	// An array of strings specifying which GitLab projects to mirror on Sourcegraph. Each string is a URL path and query that targets a GitLab API endpoint returning a list of projects. If the string only contains a query, then "projects" is used as the path. Examples: "?membership=true&search=foo", "groups/mygroup/projects".
	//
	// The special string "none" can be used as the only element to disable this feature. Projects matched by multiple query strings are only imported once. Here are a few endpoints that return a list of projects: https://docs.gitlab.com/ee/api/projects.html#list-all-projects, https://docs.gitlab.com/ee/api/groups.html#list-a-groups-projects, https://docs.gitlab.com/ee/api/search.html#scope-projects.
	"projectQuery": [
		"none"
	],
	// Other example values:
	// - [
	//     "?membership=true\u0026search=foo",
	//     "groups/mygroup/projects"
	//   ]

	// A list of projects to mirror from this GitLab instance. Supports including by name ({"name": "group/name"}) or by ID ({"id": 42}).
	"projects": null,
	// Other example values:
	// - [
	//     {
	//       "name": "group/name"
	//     },
	//     {
	//       "id": 42
	//     }
	//   ]
	// - [
	//     {
	//       "name": "gnachman/iterm2"
	//     },
	//     {
	//       "name": "gitlab-org/gitlab-ce"
	//     }
	//   ]

	// Rate limit applied when making background API requests to GitLab.
	"rateLimit": {
		"enabled": true,
		"requestsPerHour": 36000
	},

	// The pattern used to generate a the corresponding Sourcegraph repository name for a GitLab project. In the pattern, the variable "{host}" is replaced with the GitLab URL's host (such as gitlab.example.com), and "{pathWithNamespace}" is replaced with the GitLab project's "namespace/path" (such as "myteam/myproject").
	//
	// For example, if your GitLab is https://gitlab.example.com and your Sourcegraph is https://src.example.com, then a repositoryPathPattern of "{host}/{pathWithNamespace}" would mean that a GitLab project at https://gitlab.example.com/myteam/myproject is available on Sourcegraph at https://src.example.com/gitlab.example.com/myteam/myproject.
	//
	// It is important that the Sourcegraph repository name generated with this pattern be unique to this code host. If different code hosts generate repository names that collide, Sourcegraph's behavior is undefined.
	"repositoryPathPattern": "{host}/{pathWithNamespace}",

	// A GitLab access token with "api" scope. If you are enabling permissions with identity provider type "external", this token should also have "sudo" scope.
	"token": null,

	// URL of a GitLab instance, such as https://gitlab.example.com or (for GitLab.com) https://gitlab.com.
	"url": null,
	// Other example values:
	// - "https://gitlab.com"
	// - "https://gitlab.example.com"

	// An array of webhook configurations
	"webhooks": null
}

Native integration

To provide out-of-the-box code intelligence and navigation features to your users on GitLab, you will need to configure your GitLab instance. If you are using an HTTPS connection to GitLab, you will need to configure HTTPS for your Sourcegraph instance.

The Sourcegraph instance's site admin must update the corsOrigin site config property to allow the GitLab instance to communicate with the Sourcegraph instance. For example:

{
  // ...
  "corsOrigin":
    "https://my-gitlab.example.com"
  // ...
}

The site admin should also set alerts.codeHostIntegrationMessaging in global settings to ensure informational content for users in the Sourcegraph webapp references the native integration and not the browser extension.

{
  // ...
  "alerts.codeHostIntegrationMessaging": "native-integration"
  // ...
}

Access token scopes

Sourcegraph requires an access token with api permissions (and sudo, if you are using an external identity provider type). These permissions are required for the following reasons:

We are actively collaborating with GitLab to improve our integration (e.g. the Sourcegraph GitLab native integration and better APIs for querying repository permissions).

Request Type Required GitLab scope Sourcegraph usage
GET /projects api (1) For repository discovery when specifying projectQuery in code host configuration; (2) If using an external identity provider type, also used as a test query to ensure token is sudo (sudo not required otherwise).
GET /users read_user or api If you are using an external identity provider type, used to discover user accounts.
GET /users/:id read_user or api If using GitLab OAuth, used to fetch user metadata during the OAuth sign in process.
GET /projects/:id api (1) If using GitLab OAuth and repository permissions, used to determine if a user has access to a given project; (2) Used to query repository metadata (e.g. description) for display on Sourcegraph.
GET /projects/:id/repository/tree api If using GitLab OAuth and repository permissions, used to verify a given user has access to the file contents of a repository within a project (i.e. does not merely have Guest permissions).
Campaigns requests api, read_repository, write_repository Campaigns require write access to push commits and create, update and close merge requests on GitLab repositories. See "Code host interactions in campaigns" for details.

Webhooks

The webhooks setting allows specifying the webhook secrets necessary to authenticate incoming webhook requests to /.api/gitlab-webhooks.

"webhooks": [
  {"secret": "verylongrandomsecret"}
]

Using webhooks is highly recommended when using campaigns, since they speed up the syncing of pull request data between GitLab and Sourcegraph and make it more efficient.

To set up webhooks:

  1. In Sourcegraph, go to Site admin > Manage repositories and edit the GitLab configuration.
  2. Add the "webhooks" property to the configuration (you can generate a secret with openssl rand -hex 32):
    "webhooks": [{"secret": "verylongrandomsecret"}]
  3. Click Update repositories.
  4. Copy the webhook URL displayed below the Update repositories button.
  5. On GitLab, go to your project, and then Settings > Webhooks (or Settings > Integration on older GitLab versions that don't have the Webhooks option).
  6. Fill in the webhook form:
    • URL: the URL you copied above from Sourcegraph.
    • Secret token: the secret token you configured Sourcegraph to use above.
    • Trigger: select Merge request events and Pipeline events.
    • Enable SSL verification: ensure this is enabled if you have configured SSL with a valid certificate in your Sourcegraph instance.
  7. Click Add webhook.
  8. Confirm that the new webhook is listed below Project Hooks.

Done! Sourcegraph will now receive webhook events from GitLab and use them to sync merge request events, used by campaigns, faster and more efficiently.