Usage with vitest?

[joeyjiron06]

Describe the bug

I know vitest has coverage built in by default using istanbul, but I'd like to skip that built in coverage and use this plugin with vitest for some other issues i'm facing.

Are there any recommended approaches to use this with vitest?

Long description on my problem

I'm using playwright to run end to end tests in the browser and using this plugin to get code coverage which works great!
I'm using vitest to run unit tests in Node.js which allows you to collect coverage as well.
When I merge the code coverage between e2e tests and unit test coverage, i get some really strange behavior (like statements showing as not covered even though I'm certain they are covered by either unit or e2e tests). I have verified this by looking at the raw coverage-final.json output by both test suites.

I did some digging and found out that the way vitest instruments it's code with istanbul is slightly different than the way this plugin does it.

I want to try to disable vitest code coverage and use this plugin's instrumentation so that my code during unit testing gets instrumented the same way that my e2e tests does. Vitest does not provide a way to configure the options in the instrumentor, so that's why i think if i could use this plugin for instrumenting my code, i wouldn't have problems during merging.

Reproduction

https://x.com/joeyjiron06

Steps to reproduce

No response

System Info

System:
    OS: Windows 11 10.0.22631
    CPU: (24) x64 AMD Ryzen 9 5900X 12-Core Processor
    Memory: 11.92 GB / 31.90 GB
  Binaries:
    Node: 20.10.0 - ~\AppData\Local\pnpm\node.EXE
    npm: 10.2.3 - ~\AppData\Local\pnpm\npm.CMD
    pnpm: 8.6.0 - ~\AppData\Local\pnpm\pnpm.EXE
  Browsers:
    Edge: Chromium (127.0.2651.74)
    Internet Explorer: 11.0.22621.3527
  npmPackages:
    vite: ^5.3.1 => 5.4.0
    vite-plugin-istanbul: ^6.0.2 => 6.0.2

Used Package Manager

pnpm

Logs

No response

Validations

• Read the Contributing Guidelines.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Make sure this is a vite-plugin-istanbul issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to vuejs/core instead.
• The provided reproduction is a minimal reproducible example of the bug.

[iFaxity]

Hi @joeyjiron06, thanks for the bug report.

Yeah it could be good to figure out what options are necessary, or otherwise just expose most of the options for overrides by the user themselves if they need anything specific.

I'm also going look into a long sought after feature to add uncovered files into the finished coverage. With Vitests code i think i can make something similar.

Could you try out the config by cloning the repo and chancing the options by hand?
I'd be happy to test myself if i got a repo to test some different options to see if they solve the problem. Overall the coverage is hard to get to play nice, so it will be a bunch of trial and error.

[joeyjiron06]

Hi @iFaxity, I agree that trying to align the coverage to play nice can be pretty hard. So, rather than trying to do that I was hoping to skip vitest built in code coverage and use this plugin, so that the coverage is the same for both my e2e test and vitest tests.

I tried nyc vitest with these plugins in my vite config and i got an empty json coverage file

  plugins: [
    postcssLit(),
    istanbul({
      include: 'src/*',
      exclude: ['**/*.test.*', '**/*.spec.*'],
      extension: ['.js', '.ts'],
      requireEnv: false,
      forceBuildInstrument: true,
    }),
  ]

Any ideas why this plugin didn't work?

[iFaxity]

@joeyjiron06, honestly. No clue.

Best bet is that some plugins vitest injects somehow interfere with the istanbul instrumentation. Could be that it overrides whatever code is mangled with the instrumentation.

Do you see any messages from the plugin in the logs? If not then the plugin does not even get loaded.

You could try to clone the repo and link it locally to debug it. You should be able to run it and debug it, but printing out whether the plugin even loads could be helpfu.

[iFaxity]

@joeyjiron06 so i looked into how vitest collects the coverage, it uses a custom variable for storing the coverage. As this plugin doesn't expose such a variable it is not possible yet to. Also i'm pretty sure it forces its own instrumentation. I don't really see a way to deactivate this. As it is implemented together with the coverage plugin.