Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

The scraper API calls keep returning 400 error. #74

Open
bwail opened this issue Apr 9, 2022 · 0 comments
Open

The scraper API calls keep returning 400 error. #74

bwail opened this issue Apr 9, 2022 · 0 comments

Comments

@bwail
Copy link

bwail commented Apr 9, 2022 •
edited
Loading

Is this scraper no longer maintained? It relies on the tripadvisor API which is forbidden without a proper token.

Here are the logs from APIFY

2022-04-09T14:25:21.681Z ACTOR: Pulling Docker image from repository.
2022-04-09T14:25:22.895Z ACTOR: Creating Docker container.
2022-04-09T14:25:22.926Z ACTOR: Starting Docker container.
2022-04-09T14:25:26.230Z INFO  System info {"apifyVersion":"2.2.2","apifyClientVersion":"2.2.0","osType":"Linux","nodeVersion":"v16.14.0"}
2022-04-09T14:25:26.335Z INFO  Input validation OK
2022-04-09T14:25:26.670Z INFO  BasicCrawler:AutoscaledPool: state {"currentConcurrency":0,"desiredConcurrency":2,"systemStatus":{"isSystemIdle":true,"memInfo":{"isOverloaded":false,"limitRatio":0.2,"actualRatio":null},"eventLoopInfo":{"isOverloaded":false,"limitRatio":0.6,"actualRatio":null},"cpuInfo":{"isOverloaded":false,"limitRatio":0.4,"actualRatio":null},"clientInfo":{"isOverloaded":false,"limitRatio":0.3,"actualRatio":null}}}
2022-04-09T14:25:28.282Z WARN  Could not create create for session due to: Proxy responded with 400: 155 bytes {"stack":"RequestError: Proxy responded with 400: 155 bytes\n    at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)\n    at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)\n    at Object.onceWrapper (node:events:640:26)\n    at ClientRequest.emit (node:events:520:28)\n    at Socket.socketOnData (node:_http_client:522:11)\n    at Socket.emit (node:events:520:28)\n    at addChunk (node:internal/streams/readable:315:12)\n    at readableAddChunk (node:internal/streams/readable:289:9)\n    at Socket.Readable.push (node:internal/streams/readable:228:10)\n    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)\n    at TCP.callbackTra... [line-too-long]
2022-04-09T14:25:28.294Z ERROR BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.tripadvisor.com/","retryCount":1,"id":"nMLwWw9hmTQOlju"}
2022-04-09T14:25:28.298Z   RequestError: Proxy responded with 400: 155 bytes
2022-04-09T14:25:28.300Z       at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)
2022-04-09T14:25:28.302Z       at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)
2022-04-09T14:25:28.304Z       at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-04-09T14:25:28.306Z       at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)
2022-04-09T14:25:28.307Z       at Object.onceWrapper (node:events:640:26)
2022-04-09T14:25:28.309Z       at ClientRequest.emit (node:events:520:28)
2022-04-09T14:25:28.311Z       at Socket.socketOnData (node:_http_client:522:11)
2022-04-09T14:25:28.313Z       at Socket.emit (node:events:520:28)
2022-04-09T14:25:28.315Z       at addChunk (node:internal/streams/readable:315:12)
2022-04-09T14:25:28.317Z       at readableAddChunk (node:internal/streams/readable:289:9)
2022-04-09T14:25:28.319Z       at Socket.Readable.push (node:internal/streams/readable:228:10)
2022-04-09T14:25:28.321Z       at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
2022-04-09T14:25:28.323Z       at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
2022-04-09T14:25:31.561Z WARN  Could not create create for session due to: Proxy responded with 400: 155 bytes {"stack":"RequestError: Proxy responded with 400: 155 bytes\n    at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)\n    at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)\n    at Object.onceWrapper (node:events:640:26)\n    at ClientRequest.emit (node:events:520:28)\n    at Socket.socketOnData (node:_http_client:522:11)\n    at Socket.emit (node:events:520:28)\n    at addChunk (node:internal/streams/readable:315:12)\n    at readableAddChunk (node:internal/streams/readable:289:9)\n    at Socket.Readable.push (node:internal/streams/readable:228:10)\n    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)\n    at TCP.callbackTra... [line-too-long]
2022-04-09T14:25:31.577Z ERROR BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.tripadvisor.com/","retryCount":2,"id":"nMLwWw9hmTQOlju"}
2022-04-09T14:25:31.580Z   RequestError: Proxy responded with 400: 155 bytes
2022-04-09T14:25:31.582Z       at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)
2022-04-09T14:25:31.584Z       at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)
2022-04-09T14:25:31.587Z       at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-04-09T14:25:31.589Z       at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)
2022-04-09T14:25:31.591Z       at Object.onceWrapper (node:events:640:26)
2022-04-09T14:25:31.593Z       at ClientRequest.emit (node:events:520:28)
2022-04-09T14:25:31.595Z       at Socket.socketOnData (node:_http_client:522:11)
2022-04-09T14:25:31.597Z       at Socket.emit (node:events:520:28)
2022-04-09T14:25:31.599Z       at addChunk (node:internal/streams/readable:315:12)
2022-04-09T14:25:31.601Z       at readableAddChunk (node:internal/streams/readable:289:9)
2022-04-09T14:25:31.604Z       at Socket.Readable.push (node:internal/streams/readable:228:10)
2022-04-09T14:25:31.606Z       at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
2022-04-09T14:25:31.608Z       at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
2022-04-09T14:25:34.705Z WARN  Could not create create for session due to: Proxy responded with 400: 155 bytes {"stack":"RequestError: Proxy responded with 400: 155 bytes\n    at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)\n    at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)\n    at Object.onceWrapper (node:events:640:26)\n    at ClientRequest.emit (node:events:520:28)\n    at Socket.socketOnData (node:_http_client:522:11)\n    at Socket.emit (node:events:520:28)\n    at addChunk (node:internal/streams/readable:315:12)\n    at readableAddChunk (node:internal/streams/readable:289:9)\n    at Socket.Readable.push (node:internal/streams/readable:228:10)\n    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)\n    at TCP.callbackTra... [line-too-long]
2022-04-09T14:25:34.714Z ERROR BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.tripadvisor.com/","retryCount":3,"id":"nMLwWw9hmTQOlju"}
2022-04-09T14:25:34.716Z   RequestError: Proxy responded with 400: 155 bytes
2022-04-09T14:25:34.718Z       at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)
2022-04-09T14:25:34.720Z       at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)
2022-04-09T14:25:34.722Z       at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-04-09T14:25:34.724Z       at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)
2022-04-09T14:25:34.726Z       at Object.onceWrapper (node:events:640:26)
2022-04-09T14:25:34.728Z       at ClientRequest.emit (node:events:520:28)
2022-04-09T14:25:34.730Z       at Socket.socketOnData (node:_http_client:522:11)
2022-04-09T14:25:34.732Z       at Socket.emit (node:events:520:28)
2022-04-09T14:25:34.734Z       at addChunk (node:internal/streams/readable:315:12)
2022-04-09T14:25:34.736Z       at readableAddChunk (node:internal/streams/readable:289:9)
2022-04-09T14:25:34.739Z       at Socket.Readable.push (node:internal/streams/readable:228:10)
2022-04-09T14:25:34.741Z       at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
2022-04-09T14:25:34.743Z       at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
2022-04-09T14:25:37.877Z WARN  Could not create create for session due to: Proxy responded with 400: 155 bytes {"stack":"RequestError: Proxy responded with 400: 155 bytes\n    at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)\n    at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)\n    at Object.onceWrapper (node:events:640:26)\n    at ClientRequest.emit (node:events:520:28)\n    at Socket.socketOnData (node:_http_client:522:11)\n    at Socket.emit (node:events:520:28)\n    at addChunk (node:internal/streams/readable:315:12)\n    at readableAddChunk (node:internal/streams/readable:289:9)\n    at Socket.Readable.push (node:internal/streams/readable:228:10)\n    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)\n    at TCP.callbackTra... [line-too-long]
2022-04-09T14:25:37.890Z ERROR BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.tripadvisor.com/","retryCount":4,"id":"nMLwWw9hmTQOlju"}
2022-04-09T14:25:37.892Z   RequestError: Proxy responded with 400: 155 bytes
2022-04-09T14:25:37.895Z       at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)
2022-04-09T14:25:37.899Z       at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)
2022-04-09T14:25:37.901Z       at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-04-09T14:25:37.904Z       at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)
2022-04-09T14:25:37.906Z       at Object.onceWrapper (node:events:640:26)
2022-04-09T14:25:37.909Z       at ClientRequest.emit (node:events:520:28)
2022-04-09T14:25:37.913Z       at Socket.socketOnData (node:_http_client:522:11)
2022-04-09T14:25:37.915Z       at Socket.emit (node:events:520:28)
2022-04-09T14:25:37.918Z       at addChunk (node:internal/streams/readable:315:12)
2022-04-09T14:25:37.921Z       at readableAddChunk (node:internal/streams/readable:289:9)
2022-04-09T14:25:37.923Z       at Socket.Readable.push (node:internal/streams/readable:228:10)
2022-04-09T14:25:37.925Z       at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
2022-04-09T14:25:37.927Z       at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
2022-04-09T14:26:12.037Z WARN  Could not create create for session due to: Proxy responded with 400: 155 bytes {"stack":"RequestError: Proxy responded with 400: 155 bytes\n    at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)\n    at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)\n    at Object.onceWrapper (node:events:640:26)\n    at ClientRequest.emit (node:events:520:28)\n    at Socket.socketOnData (node:_http_client:522:11)\n    at Socket.emit (node:events:520:28)\n    at addChunk (node:internal/streams/readable:315:12)\n    at readableAddChunk (node:internal/streams/readable:289:9)\n    at Socket.Readable.push (node:internal/streams/readable:228:10)\n    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)\n    at TCP.callbackTra... [line-too-long]
2022-04-09T14:26:12.046Z ERROR BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.tripadvisor.com/","retryCount":5,"id":"nMLwWw9hmTQOlju"}
2022-04-09T14:26:12.048Z   RequestError: Proxy responded with 400: 155 bytes
2022-04-09T14:26:12.050Z       at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)
2022-04-09T14:26:12.052Z       at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)
2022-04-09T14:26:12.054Z       at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-04-09T14:26:12.056Z       at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)
2022-04-09T14:26:12.058Z       at Object.onceWrapper (node:events:640:26)
2022-04-09T14:26:12.060Z       at ClientRequest.emit (node:events:520:28)
2022-04-09T14:26:12.062Z       at Socket.socketOnData (node:_http_client:522:11)
2022-04-09T14:26:12.065Z       at Socket.emit (node:events:520:28)
2022-04-09T14:26:12.067Z       at addChunk (node:internal/streams/readable:315:12)
2022-04-09T14:26:12.070Z       at readableAddChunk (node:internal/streams/readable:289:9)
2022-04-09T14:26:12.072Z       at Socket.Readable.push (node:internal/streams/readable:228:10)
2022-04-09T14:26:12.075Z       at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
2022-04-09T14:26:12.077Z       at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
2022-04-09T14:26:26.675Z INFO  BasicCrawler:AutoscaledPool: state {"currentConcurrency":0,"desiredConcurrency":3,"systemStatus":{"isSystemIdle":true,"memInfo":{"isOverloaded":false,"limitRatio":0.2,"actualRatio":0},"eventLoopInfo":{"isOverloaded":false,"limitRatio":0.6,"actualRatio":0},"cpuInfo":{"isOverloaded":false,"limitRatio":0.4,"actualRatio":0},"clientInfo":{"isOverloaded":false,"limitRatio":0.3,"actualRatio":0}}}
2022-04-09T14:26:26.723Z INFO  Statistics: BasicCrawler request statistics: {"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":0,"requestsTotal":0,"crawlerRuntimeMillis":60111,"retryHistogram":[]}
2022-04-09T14:26:34.587Z WARN  Could not create create for session due to: Proxy responded with 400: 155 bytes {"stack":"RequestError: Proxy responded with 400: 155 bytes\n    at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)\n    at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)\n    at Object.onceWrapper (node:events:640:26)\n    at ClientRequest.emit (node:events:520:28)\n    at Socket.socketOnData (node:_http_client:522:11)\n    at Socket.emit (node:events:520:28)\n    at addChunk (node:internal/streams/readable:315:12)\n    at readableAddChunk (node:internal/streams/readable:289:9)\n    at Socket.Readable.push (node:internal/streams/readable:228:10)\n    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)\n    at TCP.callbackTra... [line-too-long]
2022-04-09T14:26:34.594Z ERROR BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.tripadvisor.com/","retryCount":6,"id":"nMLwWw9hmTQOlju"}
2022-04-09T14:26:34.597Z   RequestError: Proxy responded with 400: 155 bytes
2022-04-09T14:26:34.598Z       at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)
2022-04-09T14:26:34.600Z       at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)
2022-04-09T14:26:34.602Z       at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-04-09T14:26:34.604Z       at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)
2022-04-09T14:26:34.606Z       at Object.onceWrapper (node:events:640:26)
2022-04-09T14:26:34.608Z       at ClientRequest.emit (node:events:520:28)
2022-04-09T14:26:34.611Z       at Socket.socketOnData (node:_http_client:522:11)
2022-04-09T14:26:34.613Z       at Socket.emit (node:events:520:28)
2022-04-09T14:26:34.615Z       at addChunk (node:internal/streams/readable:315:12)
2022-04-09T14:26:34.617Z       at readableAddChunk (node:internal/streams/readable:289:9)
2022-04-09T14:26:34.618Z       at Socket.Readable.push (node:internal/streams/readable:228:10)
2022-04-09T14:26:34.620Z       at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
2022-04-09T14:26:34.622Z       at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
2022-04-09T14:26:37.924Z WARN  Could not create create for session due to: Proxy responded with 400: 155 bytes {"stack":"RequestError: Proxy responded with 400: 155 bytes\n    at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)\n    at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)\n    at Object.onceWrapper (node:events:640:26)\n    at ClientRequest.emit (node:events:520:28)\n    at Socket.socketOnData (node:_http_client:522:11)\n    at Socket.emit (node:events:520:28)\n    at addChunk (node:internal/streams/readable:315:12)\n    at readableAddChunk (node:internal/streams/readable:289:9)\n    at Socket.Readable.push (node:internal/streams/readable:228:10)\n    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)\n    at TCP.callbackTra... [line-too-long]
2022-04-09T14:26:37.935Z ERROR BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.tripadvisor.com/","retryCount":7,"id":"nMLwWw9hmTQOlju"}
2022-04-09T14:26:37.940Z   RequestError: Proxy responded with 400: 155 bytes
2022-04-09T14:26:37.943Z       at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)
2022-04-09T14:26:37.945Z       at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)
2022-04-09T14:26:37.947Z       at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-04-09T14:26:37.949Z       at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)
2022-04-09T14:26:37.952Z       at Object.onceWrapper (node:events:640:26)
2022-04-09T14:26:37.954Z       at ClientRequest.emit (node:events:520:28)
2022-04-09T14:26:37.956Z       at Socket.socketOnData (node:_http_client:522:11)
2022-04-09T14:26:37.959Z       at Socket.emit (node:events:520:28)
2022-04-09T14:26:37.961Z       at addChunk (node:internal/streams/readable:315:12)
2022-04-09T14:26:37.963Z       at readableAddChunk (node:internal/streams/readable:289:9)
2022-04-09T14:26:37.966Z       at Socket.Readable.push (node:internal/streams/readable:228:10)
2022-04-09T14:26:37.968Z       at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
2022-04-09T14:26:37.970Z       at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
2022-04-09T14:26:41.050Z WARN  Could not create create for session due to: Proxy responded with 400: 155 bytes {"stack":"RequestError: Proxy responded with 400: 155 bytes\n    at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)\n    at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)\n    at Object.onceWrapper (node:events:640:26)\n    at ClientRequest.emit (node:events:520:28)\n    at Socket.socketOnData (node:_http_client:522:11)\n    at Socket.emit (node:events:520:28)\n    at addChunk (node:internal/streams/readable:315:12)\n    at readableAddChunk (node:internal/streams/readable:289:9)\n    at Socket.Readable.push (node:internal/streams/readable:228:10)\n    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)\n    at TCP.callbackTra... [line-too-long]
2022-04-09T14:26:41.057Z ERROR BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.tripadvisor.com/","retryCount":8,"id":"nMLwWw9hmTQOlju"}
2022-04-09T14:26:41.060Z   RequestError: Proxy responded with 400: 155 bytes
2022-04-09T14:26:41.062Z       at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)
2022-04-09T14:26:41.064Z       at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)
2022-04-09T14:26:41.066Z       at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-04-09T14:26:41.068Z       at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)
2022-04-09T14:26:41.070Z       at Object.onceWrapper (node:events:640:26)
2022-04-09T14:26:41.072Z       at ClientRequest.emit (node:events:520:28)
2022-04-09T14:26:41.074Z       at Socket.socketOnData (node:_http_client:522:11)
2022-04-09T14:26:41.076Z       at Socket.emit (node:events:520:28)
2022-04-09T14:26:41.078Z       at addChunk (node:internal/streams/readable:315:12)
2022-04-09T14:26:41.080Z       at readableAddChunk (node:internal/streams/readable:289:9)
2022-04-09T14:26:41.082Z       at Socket.Readable.push (node:internal/streams/readable:228:10)
2022-04-09T14:26:41.084Z       at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
2022-04-09T14:26:41.086Z       at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
2022-04-09T14:26:44.198Z WARN  Could not create create for session due to: Proxy responded with 400: 155 bytes {"stack":"RequestError: Proxy responded with 400: 155 bytes\n    at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)\n    at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)\n    at Object.onceWrapper (node:events:640:26)\n    at ClientRequest.emit (node:events:520:28)\n    at Socket.socketOnData (node:_http_client:522:11)\n    at Socket.emit (node:events:520:28)\n    at addChunk (node:internal/streams/readable:315:12)\n    at readableAddChunk (node:internal/streams/readable:289:9)\n    at Socket.Readable.push (node:internal/streams/readable:228:10)\n    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)\n    at TCP.callbackTra... [line-too-long]
2022-04-09T14:26:44.206Z ERROR BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.tripadvisor.com/","retryCount":9,"id":"nMLwWw9hmTQOlju"}
2022-04-09T14:26:44.208Z   RequestError: Proxy responded with 400: 155 bytes
2022-04-09T14:26:44.210Z       at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)
2022-04-09T14:26:44.212Z       at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)
2022-04-09T14:26:44.214Z       at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-04-09T14:26:44.216Z       at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)
2022-04-09T14:26:44.218Z       at Object.onceWrapper (node:events:640:26)
2022-04-09T14:26:44.220Z       at ClientRequest.emit (node:events:520:28)
2022-04-09T14:26:44.222Z       at Socket.socketOnData (node:_http_client:522:11)
2022-04-09T14:26:44.225Z       at Socket.emit (node:events:520:28)
2022-04-09T14:26:44.227Z       at addChunk (node:internal/streams/readable:315:12)
2022-04-09T14:26:44.229Z       at readableAddChunk (node:internal/streams/readable:289:9)
2022-04-09T14:26:44.231Z       at Socket.Readable.push (node:internal/streams/readable:228:10)
2022-04-09T14:26:44.233Z       at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
2022-04-09T14:26:44.235Z       at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
2022-04-09T14:26:47.414Z WARN  Could not create create for session due to: Proxy responded with 400: 155 bytes {"stack":"RequestError: Proxy responded with 400: 155 bytes\n    at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)\n    at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)\n    at Object.onceWrapper (node:events:640:26)\n    at ClientRequest.emit (node:events:520:28)\n    at Socket.socketOnData (node:_http_client:522:11)\n    at Socket.emit (node:events:520:28)\n    at addChunk (node:internal/streams/readable:315:12)\n    at readableAddChunk (node:internal/streams/readable:289:9)\n    at Socket.Readable.push (node:internal/streams/readable:228:10)\n    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)\n    at TCP.callbackTra... [line-too-long]
2022-04-09T14:26:47.423Z ERROR BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://www.tripadvisor.com/","retryCount":10,"id":"nMLwWw9hmTQOlju"}
2022-04-09T14:26:47.426Z   RequestError: Proxy responded with 400: 155 bytes
2022-04-09T14:26:47.428Z       at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)
2022-04-09T14:26:47.430Z       at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)
2022-04-09T14:26:47.433Z       at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-04-09T14:26:47.443Z       at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)
2022-04-09T14:26:47.446Z       at Object.onceWrapper (node:events:640:26)
2022-04-09T14:26:47.448Z       at ClientRequest.emit (node:events:520:28)
2022-04-09T14:26:47.450Z       at Socket.socketOnData (node:_http_client:522:11)
2022-04-09T14:26:47.452Z       at Socket.emit (node:events:520:28)
2022-04-09T14:26:47.454Z       at addChunk (node:internal/streams/readable:315:12)
2022-04-09T14:26:47.456Z       at readableAddChunk (node:internal/streams/readable:289:9)
2022-04-09T14:26:47.457Z       at Socket.Readable.push (node:internal/streams/readable:228:10)
2022-04-09T14:26:47.459Z       at TCP.onStreamRead (node:internal/stream_base_commons:190:23)
2022-04-09T14:26:47.461Z       at TCP.callbackTrampoline (node:internal/async_hooks:130:17)
2022-04-09T14:26:50.973Z WARN  Could not create create for session due to: Proxy responded with 400: 155 bytes {"stack":"RequestError: Proxy responded with 400: 155 bytes\n    at Request._beforeError (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:333:21)\n    at Request.flush (/usr/src/app/node_modules/got-cjs/dist/source/core/index.js:322:18)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at ClientRequest.<anonymous> (/usr/src/app/node_modules/got-scraping/dist/resolve-protocol.js:37:28)\n    at Object.onceWrapper (node:events:640:26)\n    at ClientRequest.emit (node:events:520:28)\n    at Socket.socketOnData (node:_http_client:522:11)\n    at Socket.emit (node:events:520:28)\n    at addChunk (node:internal/streams/readable:315:12)\n    at readableAddChunk (node:internal/streams/readable:289:9)\n    at Socket.Readable.push (node:internal/streams/readable:228:10)\n    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)\n    at TCP.callbackTra... [line-too-long]
2022-04-09T14:26:51.009Z INFO  Request https://www.tripadvisor.com/ failed too many times
2022-04-09T14:26:51.152Z INFO  BasicCrawler: All the requests from request list and/or request queue have been processed, the crawler will shut down.
2022-04-09T14:26:51.304Z INFO  BasicCrawler: Final request statistics: {"requestsFinished":0,"requestsFailed":1,"retryHistogram":[null,null,null,null,null,null,null,null,null,null,1],"requestAvgFailedDurationMillis":35,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":35,"requestsTotal":1,"crawlerRuntimeMillis":84691}
2022-04-09T14:26:51.307Z INFO  Requests failed: 1
2022-04-09T14:26:51.309Z INFO  Crawler finished.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@bwail