Issue with NPM packages in the Github registry

[alexander-chi]

Select Topic Area

Question

Body

Hi there!

I wanted to check if anyone else was experiencing issues pulling packages from private registries on Github.

Just this evening our CI/CD started failing and now on NPM install im getting errors like this

npm WARN tarball tarball data for @{{my-org}}/{{my-package}}@file:/download/@{{my-org}}/{{my-package}}/1.0.34/8ab6ba8e18aa9b6550563d3251efe97007dc6e7f (sha512-5h/wuPQwwlEnH489BwT4YNBt+Q2Kb5olzxkNwV8MESSVQ4DVYwm9sANKbf9ILWFnrpMnf5httwRDQM7MZ3bPCg==) seems to be corrupted. Trying again.

npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path /download/@{{my-org}}/{{my-package}}/1.0.34/8ab6ba8e18aa9b6550563d3251efe97007dc6e7f/package.json
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, open '/download/@{{my-org}}/{{my-package}}/1.0.34/8ab6ba8e18aa9b6550563d3251efe97007dc6e7f/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent 

Ive tried:

• Removing package-lock
• Removing node_modules
• Updating node & npm

Its happening in CI/CD too, and in different repos

Theres definetly an issue fetching packages

2284 http fetch GET 200 https://npm.pkg.github.com/@urbanchain%2fcdk-utils 7104ms (cache miss)

It's taking 7s to fetch a package that's only a few mb

2093 http fetch GET 200 https://npm.pkg.github.com/@urbanchain%2fhub-types 6391ms (cache miss)
2170 http fetch GET 200 https://npm.pkg.github.com/@urbanchain%2fhub-client 6578ms (cache miss)

...

2428 verbose stack TypeError: Wrong spec type (directory) for RemoteFetcher. Supported types: remote
2428 verbose stack     at [_assertType] (/opt/homebrew/lib/node_modules/npm/node_modules/pacote/lib/fetcher.js:271:13)
2428 verbose stack     at new FetcherBase (/opt/homebrew/lib/node_modules/npm/node_modules/pacote/lib/fetcher.js:59:22)
2428 verbose stack     at new RemoteFetcher (/opt/homebrew/lib/node_modules/npm/node_modules/pacote/lib/remote.js:12:5)
2428 verbose stack     at [pacote.Fetcher._tarballFromResolved] (/opt/homebrew/lib/node_modules/npm/node_modules/pacote/lib/registry.js:354:12)
2428 verbose stack     at retry.retries (/opt/homebrew/lib/node_modules/npm/node_modules/pacote/lib/fetcher.js:331:64)
2428 verbose stack     at /opt/homebrew/lib/node_modules/npm/node_modules/promise-retry/index.js:29:24
2429 verbose cwd /Users/alex/Git/internal-monorepo

[Anupheaus]

I'm getting the same thing, a package created 4 hours ago works, but any created recently give the same error, both in npm and pnpm - I thought I was going mad rolling back changes etc, but I definitely think now it is Github Package registry related.

[alexander-chi]

Oh well i'm glad to hear it's not just me!

Nothing on the status page

2428 verbose stack TypeError: Wrong spec type (directory) for RemoteFetcher. Supported types: remote
2428 verbose stack     at [_assertType] (/opt/homebrew/lib/node_modules/npm/node_modules/pacote/lib/fetcher.js:271:13)
2428 verbose stack     at new FetcherBase (/opt/homebrew/lib/node_modules/npm/node_modules/pacote/lib/fetcher.js:59:22)
2428 verbose stack     at new RemoteFetcher (/opt/homebrew/lib/node_modules/npm/node_modules/pacote/lib/remote.js:12:5)
2428 verbose stack     at [pacote.Fetcher._tarballFromResolved] (/opt/homebrew/lib/node_modules/npm/node_modules/pacote/lib/registry.js:354:12)
2428 verbose stack     at retry.retries (/opt/homebrew/lib/node_modules/npm/node_modules/pacote/lib/fetcher.js:331:64)
2428 verbose stack     at /opt/homebrew/lib/node_modules/npm/node_modules/promise-retry/index.js:29:24
2429 verbose cwd /Users/alex/Git/internal-monorepo

[Jianrong-Yu]

Same issue here, I tried both locally and on CI, both yarn and npm, all of them have the same issue.

275 silly tarball no local data for @*****/*****@file:/download/@*****/*****/**.*.*/ididididid. Extracting by manifest.
276 warn tarball tarball data for @*****/*****@file:/download/@*****/*****/**.*.*/ididididid (sha512-*****) seems to be corrupted. Trying again.
277 warn tarball tarball data for @*****/*****@file:/download/@*****/*****/**.*.*/ididididid (sha512-*****) seems to be corrupted. Trying again.

For trouble shooting: I tried to use curl to request the .tar:

curl -H "Authorization: token ***" https://npm.pkg.github.com/download/@****/**/0.0.1/***

And the response is:

<a href="https://pkg-npm.githubusercontent.com/npmregistryv2prod/blobs/***/**/0.0.1/***?se=2024-04-09T00%3A42%3A45Z&amp;sig=***%3D&amp;sp=r&amp;spr=https&amp;sr=b&amp;sv=****-**-**">Found</a>.

[xanido]

It looks like the package tarballs are there, but the registry is returning relative tarball URLs rather than absolute.

E.g. the tarball URL returned by NPM is absolute:

$ npm view express dist.tarball
https://registry.npmjs.org/express/-/express-4.19.2.tgz

However the tarball URL returned by GH pkg registry is relative:

$ npm view @***/*** dist.tarball
/download/@***/***/x.x.x/***

You can download the tarball by prepending https://npm.pkg.github.com e.g.:

curl -L -H "Authorization: $(gh auth token)" https://npm.pkg.github.com/download/@***/***/x.x.x/*** > pkg.tar.gz

[alexander-chi]

npm pack seems to download the tarball no issue so I wonder how the url resolution is different between the two commands

[xanido]

Interesting, npm pack is not able to download these private packages correctly on my end at the moment- it's treating the tarball URL as a local file URI...

33 http fetch GET 200 https://npm.pkg.github.com/@***/*** 3596ms (cache updated)
34 http fetch GET 200 https://npm.pkg.github.com/@***/*** 340ms (cache updated)
35 silly tarball no local data for file:/download/@***/***/x.x.x/***. Extracting by manifest.
36 warn tarball tarball data for file:/download/@***/***/x.x.x/*** (sha512-***) seems to be corrupted. Trying again.

[Elecweb]

I also unable to use npm pack with private packages.

[CharlieGreenman]

yeah same here actions/checkout#1675 (comment)

[ryzr]

I have submitted a support ticket. Hopefully we get an update on https://www.githubstatus.com/ soon.

[aiell0]

Also seeing the same problems on our side. Hoping for a quick resolution.

[amedve]

Same for me.

[kobiebotha]

Same on our side

[CharlieGreenman]

Got bored + starting dancing. Would prefer not to dance and hoping it gets resolved soon: https://x.com/razroo_chief/status/1777506664947540078

[wgcorrea]

Same on our side

[oliver-euc]

pls fix fr fr

[CharlieGreenman]

Also, wild theory, not sure if related to xz backdoor, but would explain why happened: https://gist.github.com/thesamesam/223949d5a074ebc3dce9ee78baad9e27

[bretislav]

Currently experiencing the same issue. Does not reproduce on local, only via github action. Have tried re-running the action(s) and publishing a new version of a package. Error is:
npm WARN tarball tarball data for @{{my-org}}/{{my-package}}@file:/download/@{{my-org}}/{{my-package}}/1.0.3/94e8e8ee8d85ddef1fa1271941e971622bb33d48 (sha512-xcsR6ODrDA5JnPJTrQcTc9RUjHPFKMO3VC1gglbRxSdXAXbH7Bk6ySuIWqPnEakkR5enRilxVfpSLdArqt39oQ==) seems to be corrupted. Trying again.

[frankpengau]

Currently experiencing the same issue.

Seems like the packages are getting resolved to local directories?

package-lock.json

node_modules/<@org>/<pkg-name>
...
"resolved": "file:../../../../../../download/<@org>/<pkg-name>/<pkg-version>/<hash>

[ryanhinojosa]

We are experiencing the same issue on all of our repositories that are referencing our private github packages.

[changwoolab]

Same issue here

[juicyjusung]

Same here

[wjaykim]

It looks like the package tarballs are there, but the registry is returning relative tarball URLs rather than absolute.

E.g. the tarball URL returned by NPM is absolute:

$ npm view express dist.tarball
https://registry.npmjs.org/express/-/express-4.19.2.tgz

However the tarball URL returned by GH pkg registry is relative:

$ npm view @***/*** dist.tarball
/download/@***/***/x.x.x/***

You can download the tarball by prepending https://npm.pkg.github.com e.g.:

curl -L -H "Authorization: $(gh auth token)" https://npm.pkg.github.com/download/@***/***/x.x.x/*** > pkg.tar.gz

This must be the cause of this issue.

[shiba-hiro]

I faced same issue.

As a temporary workaround, I manually fill in the resolved and integrity properties in package-lock.json to get installation done.
(npm install logs told me resolved (missing https://npm.pkg.github.com host name) and valid integrity)

[jedsada-gh]

For the temporary workaround is work for me.

[dylanvorster]

same here (pretty sure this should now at least be an outage).

[liheng-zhang]

Same here

[sajsanghvi]

Truly wild its still not reflected on the status page

[zoeyliu200]

Same here

[danielxvu]

This is also happening for our org and has completely broken our build pipelines.

[ria3100]

Same for me.

[timwuhaotian]

Having the same issue for org scoped package installation!

[ryzr]

I was just able to npm install successfully for the first time today.

Github Status page now shows "We are investigating reports of degraded performance.".

EDIT: status update
"We are investigating reports of issues with downloading NPM packages." looks like it's being/been fixed :-)

[frankpengau]

Incident seems to be resolved: https://www.githubstatus.com/incidents/6tgmk1j7b4rn

Can confirm npm install is working again and able to resolve correctly to https://npm.pkg.github.com/download/<@org>/<pkg-name>/<version>/<hash> instead of file:../../../download/<@org>/<pkg-name>/<version>/<hash> in package-lock.json.

[dylanvorster]

Just a reminder you may need to run:

pnpm store prune / the equivalent for your package manager if it does local caching