haskell-te: 5d14ad62b5e571a1c7adbdc14d42e1aef0d583eb

     1: From: Chris Warburton
     2: Date: Mon, 30 Oct 2017 13:36:11 +0000
     3: State: resolved
     4: Subject: Trigger errors in benchmarking Nix
     5: Message-Id: <fc1b90cdeb0a74a4-0-artemis@nixos>
     6: resolution: fixed
     7: 
     8: When nix-eval needs some Haskell packages, it checks to see if they're
     9: available. If so, it calls GHC as-is, if not it uses Nix to generate an
    10: environment with those packages, potentially compiling them.
    11: 
    12: We don't want to be compiling during a benchmark, since it'll skew our
    13: runtimes massively upwards. We *should* be running without having to
    14: compile, since we're only using a given package (e.g. TEBenchmark,
    15: list-full, etc.), but we can't be sure.
    16: 
    17: We should set the Haskell overrides used by nix-eval to some sort of
    18: error, e.g. `abort "Benchmarking shouldn't need to make a new
    19: env"`. This way, we'll know if there's a potential skew.