Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Downloader needs to honor --concurrentThreads #180

Open
github-actions bot opened this issue Jan 3, 2025 · 1 comment
Open

Downloader needs to honor --concurrentThreads #180

github-actions bot opened this issue Jan 3, 2025 · 1 comment
Assignees
Labels
download msar download msar packages/myschoolapp-reporting

Comments

@github-actions
Copy link

github-actions bot commented Jan 3, 2025

// TODO Downloader needs to honor --concurrentThreads

import * as common from '../../common.js';
import * as Cache from './Cache.js';
import * as AuthenticatedFetch from './Downloader/AuthenticatedFetch.js';
import * as HTTPFetch from './Downloader/HTTPFetch.js';
import { Strategy } from './Downloader/Strategy.js';

export type Options = {
  host: string;
} & common.output.args.Parsed &
  common.PuppeteerSession.args.Parsed &
  common.workflow.args.Parsed;

// TODO Downloader needs to honor --concurrentThreads
export class Downloader implements Strategy {
  private auth: AuthenticatedFetch.Downloader;
  private http: HTTPFetch.Downloader;
  private host: string;

  public constructor({ host, outputOptions, ...options }: Options) {
    const { outputPath } = outputOptions;
    if (!outputPath) {
      throw new common.output.OutputError('Downloader requires outputPath');
    }
    this.host = host;
    this.auth = new AuthenticatedFetch.Downloader({
      host,
      outputOptions,
      ...options
    });
    this.http = new HTTPFetch.Downloader({ outputPath, ...options });
  }

  public async download(original: string, filename?: string) {
@battis battis added msar packages/myschoolapp-reporting download msar download labels Jan 3, 2025
@battis battis added this to the msar v0.5 milestone Jan 3, 2025
@battis battis closed this as completed in a879c8a Jan 4, 2025
@battis
Copy link
Owner

battis commented Jan 4, 2025

Not so fast… right now it's concurrent threads per section -- it would be nice to have this be global.

@battis battis reopened this Jan 4, 2025
@battis battis removed this from the msar v0.5 milestone Jan 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
download msar download msar packages/myschoolapp-reporting
Projects
None yet
Development

No branches or pull requests

1 participant