Something wonky with the lockfile? #1882

New Issue

jtojnar · 2021-12-04T04:22:16Z

jtojnar commented

2021-12-04 04:22:16 +00:00

I tried updating my frontend Nix package to latest commit but yarn2nix does not seem to like it as shown by the following reproducer:

$ nix-build vikutest.nix --show-trace
error: hash 'undefined' has wrong length for hash type 'sha1'

       … while evaluating 'escapeShellArg'

       at /home/jtojnar/Projects/nixpkgs/lib/strings.nix:318:20:

          317|   */
          318|   escapeShellArg = arg: "'${replaceStrings ["'"] ["'\\''"] (toString arg)}'";
             |                    ^
          319|

       … from call site

       at /home/jtojnar/Projects/nixpkgs/pkgs/build-support/trivial-builders.nix:420:19:

          419|           mkdir -p "$(dirname ${lib.escapeShellArg x.name})"
          420|           ln -s ${lib.escapeShellArg x.path} ${lib.escapeShellArg x.name}
             |                   ^
          421|       '') entries}

       … while evaluating anonymous lambda

       at /home/jtojnar/Projects/nixpkgs/pkgs/build-support/trivial-builders.nix:418:31:

          417|       cd $out
          418|       ${lib.concatMapStrings (x: ''
             |                               ^
          419|           mkdir -p "$(dirname ${lib.escapeShellArg x.name})"

       … from call site

       … while evaluating 'concatMapStrings'

       at /home/jtojnar/Projects/nixpkgs/lib/strings.nix:53:25:

           52|   */
           53|   concatMapStrings = f: list: concatStrings (map f list);
             |                         ^
           54|

       … from call site

       at /home/jtojnar/Projects/nixpkgs/pkgs/build-support/trivial-builders.nix:418:9:

          417|       cd $out
          418|       ${lib.concatMapStrings (x: ''
             |         ^
          419|           mkdir -p "$(dirname ${lib.escapeShellArg x.name})"

       … while evaluating the attribute 'buildCommand' of the derivation 'offline'

       at /home/jtojnar/Projects/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:205:7:

          204|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          205|       name =
             |       ^
          206|         let

       … while evaluating the attribute 'buildPhase' of the derivation 'vikunja-frontend-modules-unstable-2021-12-03'

       at /home/jtojnar/Projects/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:205:7:

          204|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          205|       name =
             |       ^
          206|         let

       … while evaluating the attribute 'configurePhase' of the derivation 'vikunja-frontend-modules'

       at /home/jtojnar/Projects/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:205:7:

          204|     // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) {
          205|       name =
             |       ^
          206|         let

vikutest.nix as follows:

let
  pkgs = import <nixpkgs> { };

  version = "unstable-2021-12-03";

  src = pkgs.fetchFromGitea {
    domain = "kolaente.dev";
    owner = "vikunja";
    repo = "frontend";
    rev = "fe5770082a725351c6703e2c8d2585aac8a86bb3";
    sha256 = "fUjrI1DFrdOWnwnORrX32zo/hinKl4/Vm8u7qzeUEZU=";
  };
  # src = /home/jtojnar/Projects/vikunja-frontend;

  frontend-modules = pkgs.mkYarnPackage rec {
    name = "vikunja-frontend-modules";
    inherit version src;

    doDist = false;
  };
in
frontend-modules

The only thing I can glean from the trace is that the offline package has something to do with it but bisecting points me to 17d791027c, which confusingly does not mention offline at all.

I tried updating my frontend Nix package to latest commit but yarn2nix does not seem to like it as shown by the following reproducer: ```ShellSession $ nix-build vikutest.nix --show-trace error: hash 'undefined' has wrong length for hash type 'sha1' … while evaluating 'escapeShellArg' at /home/jtojnar/Projects/nixpkgs/lib/strings.nix:318:20: 317| */ 318| escapeShellArg = arg: "'${replaceStrings ["'"] ["'\\''"] (toString arg)}'"; | ^ 319| … from call site at /home/jtojnar/Projects/nixpkgs/pkgs/build-support/trivial-builders.nix:420:19: 419| mkdir -p "$(dirname ${lib.escapeShellArg x.name})" 420| ln -s ${lib.escapeShellArg x.path} ${lib.escapeShellArg x.name} | ^ 421| '') entries} … while evaluating anonymous lambda at /home/jtojnar/Projects/nixpkgs/pkgs/build-support/trivial-builders.nix:418:31: 417| cd $out 418| ${lib.concatMapStrings (x: '' | ^ 419| mkdir -p "$(dirname ${lib.escapeShellArg x.name})" … from call site … while evaluating 'concatMapStrings' at /home/jtojnar/Projects/nixpkgs/lib/strings.nix:53:25: 52| */ 53| concatMapStrings = f: list: concatStrings (map f list); | ^ 54| … from call site at /home/jtojnar/Projects/nixpkgs/pkgs/build-support/trivial-builders.nix:418:9: 417| cd $out 418| ${lib.concatMapStrings (x: '' | ^ 419| mkdir -p "$(dirname ${lib.escapeShellArg x.name})" … while evaluating the attribute 'buildCommand' of the derivation 'offline' at /home/jtojnar/Projects/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:205:7: 204| // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) { 205| name = | ^ 206| let … while evaluating the attribute 'buildPhase' of the derivation 'vikunja-frontend-modules-unstable-2021-12-03' at /home/jtojnar/Projects/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:205:7: 204| // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) { 205| name = | ^ 206| let … while evaluating the attribute 'configurePhase' of the derivation 'vikunja-frontend-modules' at /home/jtojnar/Projects/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:205:7: 204| // (lib.optionalAttrs (attrs ? name || (attrs ? pname && attrs ? version)) { 205| name = | ^ 206| let ``` `vikutest.nix` as follows: ```nix let pkgs = import <nixpkgs> { }; version = "unstable-2021-12-03"; src = pkgs.fetchFromGitea { domain = "kolaente.dev"; owner = "vikunja"; repo = "frontend"; rev = "fe5770082a725351c6703e2c8d2585aac8a86bb3"; sha256 = "fUjrI1DFrdOWnwnORrX32zo/hinKl4/Vm8u7qzeUEZU="; }; # src = /home/jtojnar/Projects/vikunja-frontend; frontend-modules = pkgs.mkYarnPackage rec { name = "vikunja-frontend-modules"; inherit version src; doDist = false; }; in frontend-modules ``` The only thing I can glean from the trace is that the `offline` package has something to do with it but bisecting points me to 17d791027ccf0cc3a08a54f3a677168c4960569c, which confusingly does not mention `offline` at all.

jtojnar commented

2021-12-04 04:51:27 +00:00

The offline package is actually an artefact of yarn2nix.

I tried replicating what yarn2nix does:

let
  pkgs = import <nixpkgs> { };

  version = "unstable-2021-12-03";

  src = pkgs.fetchFromGitea {
    domain = "kolaente.dev";
    owner = "vikunja";
    repo = "frontend";
    rev = "fe5770082a725351c6703e2c8d2585aac8a86bb3";
    sha256 = "fUjrI1DFrdOWnwnORrX32zo/hinKl4/Vm8u7qzeUEZU=";
  };

  yarnLock = src + "/yarn.lock";
  yarnNix = pkgs.yarn2nix-moretea.mkYarnNix { inherit yarnLock; };
in
yarnNix

and the produced contains the following:

    {
      name = "https___registry.npmjs.org_caniuse_lite___caniuse_lite_1.0.30001265.tgz";
      path = fetchurl {
        name = "https___registry.npmjs.org_caniuse_lite___caniuse_lite_1.0.30001265.tgz";
        url  = "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001265.tgz";
        sha1 = "undefined";
      };
    }

Weirdly, if I just run yarn2nix > yarndeps.nix in the project directory, there is a hash so possibly this is a bug in yarn2nix that it did not detect failure.

The `offline` package is actually an artefact of yarn2nix. I tried replicating [what `yarn2nix` does](https://github.com/NixOS/nixpkgs/blob/bb9bd465b625bfc971908c5d3d84ce517e1c0691/pkgs/development/tools/yarn2nix-moretea/yarn2nix/default.nix#L233): ```nix let pkgs = import <nixpkgs> { }; version = "unstable-2021-12-03"; src = pkgs.fetchFromGitea { domain = "kolaente.dev"; owner = "vikunja"; repo = "frontend"; rev = "fe5770082a725351c6703e2c8d2585aac8a86bb3"; sha256 = "fUjrI1DFrdOWnwnORrX32zo/hinKl4/Vm8u7qzeUEZU="; }; yarnLock = src + "/yarn.lock"; yarnNix = pkgs.yarn2nix-moretea.mkYarnNix { inherit yarnLock; }; in yarnNix ``` and the produced contains the following: ```nix { name = "https___registry.npmjs.org_caniuse_lite___caniuse_lite_1.0.30001265.tgz"; path = fetchurl { name = "https___registry.npmjs.org_caniuse_lite___caniuse_lite_1.0.30001265.tgz"; url = "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001265.tgz"; sha1 = "undefined"; }; } ``` Weirdly, if I just run `yarn2nix > yarndeps.nix` in the project directory, there is a hash so possibly this is a bug in `yarn2nix` that it did not detect failure.

jtojnar commented

2021-12-04 04:57:48 +00:00

But the weird thing is that every sandboxed build seems to fail in the same way. Even if I switch to a different commit that does not have the same yarn.lock so this is not just non-deterministic failure that got cached (plus that would likely get revealed by nix-build --check, which it did not).

  src = pkgs.fetchFromGitea {
    domain = "kolaente.dev";
    owner = "vikunja";
    repo = "frontend";
    rev = "2041722b8a1b88eafc689ca781ad2651538bc56b";
    sha256 = "7+hbY++DcHS1GR7kurwqwm+4m3yWIQSl4D8yGgzBfC0=";
  };

But the weird thing is that every sandboxed build seems to fail in the same way. Even if I switch to a different commit that does not have the same `yarn.lock` so this is not just non-deterministic failure that got cached (plus that would likely get revealed by `nix-build --check`, which it did not). ```nix src = pkgs.fetchFromGitea { domain = "kolaente.dev"; owner = "vikunja"; repo = "frontend"; rev = "2041722b8a1b88eafc689ca781ad2651538bc56b"; sha256 = "7+hbY++DcHS1GR7kurwqwm+4m3yWIQSl4D8yGgzBfC0="; }; ```

jtojnar commented

2021-12-04 05:34:12 +00:00

I managed to simplify this to a lockfile with a single package:
f1fcab1e8d/yarn2nix-incomprehensible-error

I managed to simplify this to a lockfile with a single package: https://github.com/jtojnar/repro/tree/f1fcab1e8dd9ed774714f74966e7950fb06ac0d3/yarn2nix-incomprehensible-error

jtojnar commented

2021-12-04 05:41:37 +00:00

Turns out the hash fragment of the resolved URL is the sha1 hash of the tarball and newer versions of Yarn no longer include it according to https://github.com/NixOS/nixpkgs/pull/119522. Sorry for the noise.

Turns out the hash fragment of the `resolved` URL is the `sha1` hash of the tarball and newer versions of Yarn no longer include it according to https://github.com/NixOS/nixpkgs/pull/119522. Sorry for the noise.

jtojnar closed this issue

2021-12-04 05:41:37 +00:00

konrad commented

2021-12-04 10:24:53 +00:00

Very interesting.

dpschen commented

2021-12-04 10:52:57 +00:00

I don't know much about NixOS but I may have an idea of the problems origin:

The browserslist:update command is responsible to update the compatability tables for browserfeatures which is e.g. used for Autoprefixer.

Maybe the issue is not the linked commit but instead the added entry to the drone.yml: ed6dc94873

I don't know much about NixOS but I may have an idea of the problems origin: The `browserslist:update` command is responsible to update the compatability tables for browserfeatures which is e.g. used for Autoprefixer. Maybe the issue is not the linked commit but instead the added entry to the drone.yml: https://kolaente.dev/vikunja/frontend/commit/ed6dc948738239421b6bcd0882019f1f7730fa1c

jtojnar commented

2021-12-04 17:18:37 +00:00

It really is just that yarn2nix does not like resolved "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001265.tgz" line because it no longer contains #sha1. But that is problem with yarn2nix and needs to be fixed there. https://github.com/NixOS/nixpkgs/pull/119522 should do that.

It really is just that `yarn2nix` does not like `resolved "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001265.tgz"` line because it no longer contains `#sha1`. But that is problem with yarn2nix and needs to be fixed there. https://github.com/NixOS/nixpkgs/pull/119522 should do that.

👍 1 😆 1

konrad referenced this issue from a commit

2024-02-07 14:00:33 +00:00

chore(deps): update dependency netlify-cli to v10.1.0 (#1882)

Sign in to join this conversation.