[Self-Host] does self-host support scrape web page written by vue.js or react.js ?

[wangping886]

when i exec http://localhost:3002/v1/scrape but no content return. what is the body should set can get content
Screenshots

Environment (please complete the following information):

• Docker Version (if applicable): [e.g. 20.10.14]

[mogery]

Hi there, you need to set up the Playwright microservice to be able to scrape sites that use JavaScript.

[mogery]

Scraping via fetch deemed successful.

This means that Firecrawl is using the fetch engine, not the playwright engine.

An unexpected error happened while scraping with playwright.

Are the playwright engine and the Firecrawl .env variables configured correctly?

[wangping886]

i'm trying follow steps . i need scraping with myself playwright ?

(Optional) Running with TypeScript Playwright Service

Update the docker-compose.yml file to change the Playwright service:

    build: apps/playwright-service
TO

    build: apps/playwright-service-ts
Set the PLAYWRIGHT_MICROSERVICE_URL in your .env file:

PLAYWRIGHT_MICROSERVICE_URL=http://localhost:3000/scrape
Don't forget to set the proxy server in your .env file as needed.

[mogery]

What is your PLAYWRIGHT_MICROSERVICE_URL set to in your .env file?

[wangping886]

PLAYWRIGHT_MICROSERVICE_URL=http://playwright-service:3000/html

[mogery]

PLAYWRIGHT_MICROSERVICE_URL=http://playwright-service:3000/html

Can you try with PLAYWRIGHT_MICROSERVICE_URL=http://playwright-service:3000/scrape ?
Are you using playwright-service or playwright-service-ts?

[wangping886]

I'm docker build playwright-service-ts now . it's very slowly. this step (Optional) Running with TypeScript Playwright Service is not have to do ?

PLAYWRIGHT_MICROSERVICE_URL=http://playwright-service:3000/html can't get content. and the server no logs.

I'll use /scrape to have a try , still can't retrieve content

[wangping886]

i use playwright-service and set .env with PLAYWRIGHT_MICROSERVICE_URL=http://playwright-service:3000/scrape . don't get content

[Wanli063]

Hello, have you solved the problem? I had the same problem.

[wangping886]

Hello, have you solved the problem? I had the same problem.

no, the author don't tell me how to solve

[namhnz]

Is has error:

My Ubuntu machine is:
Distributor ID: Ubuntu
Description: Ubuntu 20.04.6 LTS
Release: 20.04
Codename: focal

[eliaozi]

Try this:

• Modify fetch.ts , add debug info in order to analyze log information(optional).

• Modify docker-compose.yaml, change playwright-service to playwright-service-ts. And change PLAYWRIGHT_MICROSERVICE_URL,add /scrape.
  

• Run docker compose build and docker compose up.

[ocampoje17]

Try this:

• Modify fetch.ts , add debug info in order to analyze log information(optional).

• Modify docker-compose.yaml, change playwright-service to playwright-service-ts. And change PLAYWRIGHT_MICROSERVICE_URL,add /scrape.
  
• Run docker compose build and docker compose up.

I tried with version 1.2.1 and I think this solution works well
And I also need to change the .env file to this: