haskell-te: 2cb87543b7b202909bcd54622d9647e18619d891

     1: From: Chris Warburton
     2: Date: Thu, 17 Aug 2017 20:04:33 +0100
     3: Subject: Re: See if our benchmark setup could be built by Nix
     4: Message-Id: <a48cf44569ff379d-90f38293a514930a-artemis@nixos>
     5: References: <a48cf44569ff379d-0-artemis@nixos>
     6: In-Reply-To: <a48cf44569ff379d-0-artemis@nixos>
     7: 
     8: I've just realised a much more obvious improvement: we provide an
     9: environment via shell.nix for running the benchmarks; this provides asv
    10: and the asv-nix plugin. Why not *also* make it provide all of the
    11: required commands, env vars, etc.?
    12: 
    13: We currently have three "levels" of Nix builder to run our benchmarks:
    14: 
    15:  1 nix-shell builds an environment, which we only use for providing asv
    16:    and asv-nix.
    17:  2 benchmarks/default.nix builds an environment, which provides the
    18:    python binary used for invoking the benchmark functions, but isn't
    19:    specific to any particular git commit.
    20:  3 asv-nix provides each particular git commit to the
    21:    benchmarks/default.nix builder, so it can put commit-specific things
    22:    into the environment. It currently ignores that though.
    23: 
    24: We don't use 3, but it's nice to keep the option open. We currently do
    25: all of the complicated stuff in 2, which gets called *inside* 1, and
    26: hence may run into problems with NIX_PATH and friends.
    27: 
    28: We should move as much as possible out of 2 and into 1. This way, we
    29: aren't building complicated stuff *inside* a Nix environment, so we can
    30: check it all with release.nix, tests.nix and friends.