Python dependencies

Site admins can sync Python packages from any Python package mirror, including open source code from pypi.org or a private mirror such as Nexus, to their Sourcegraph instance so that users can search and navigate the repositories.

To add Python dependencies to Sourcegraph you need to setup a Python dependencies code host:

1. As site admin: go to Site admin > Global settings and enable the experimental feature by adding: {"experimentalFeatures": {"pythonPackages": "enabled"} }
2. As site admin: go to Site admin > Manage code hosts
3. Select Python Dependencies.
4. Configure the connection by following the instructions above the text field. Additional fields can be added using Cmd/Ctrl+Space for auto-completion. See the configuration documentation below.
5. Press Add repositories.

Repository syncing

There are currently two ways to sync Python dependency repositories.

• Code host configuration: manually list dependencies in the "dependencies" section of the JSON configuration when creating the Python dependency code host. This method can be useful to verify that the credentials are picked up correctly without having to run a dependencies search.

Sourcegraph tries to find each dependency repository in all configured "urls" until it's found. This means you can configure a public mirror first and fallback to a private one second (e.g. "urls": ["https://pypi.org", "https://admin:[email protected]"]).

Credentials

Each entry in the "urls" array can contain basic auth if needed (e.g. https://user:[email protected]).

Rate limiting

By default, requests to the Python package mirrors will be rate-limited based on a default internal limit. (source)

"rateLimit": {
  "enabled": true,
  "requestsPerHour": 57600
}

where the requestsPerHour field is set based on your requirements.

Not recommended: Rate-limiting can be turned off entirely as well. This increases the risk of overloading the proxy.

"rateLimit": {
  "enabled": false
}

Configuration

Python dependencies code host connections support the following configuration options, which are specified in the JSON editor in the site admin "Manage code hosts" area.

admin/external_service/python-packages.schema.json

{
	// An array of strings specifying Python packages to mirror in Sourcegraph.
	"dependencies": null,
	// Other example values:
	// - [
	//     "numpy==1.22.3",
	//     "pytorch==1.0.2"
	//   ]

	// Rate limit applied when making background API requests to the configured Python simple repository APIs.
	"rateLimit": {
		"enabled": true,
		"requestsPerHour": 57600
	},

	// The list of Python simple repository URLs to fetch packages from. 404 Not found or 410 Gone responses will result in the next URL to be attempted.
	"urls": [
		"https://pypi.org/simple"
	]
	// Other example values:
	// - [
	//     "https://private.mycorp.org/simple",
	//     "https://pypi.org/simple"
	//   ]
}