Crawling Not Working - https://www.69yuedu.net/

[steve30990]

Describe the bug

Crawling on site https://www.69yuedu.net/

(Note 69shuba use to have this problem, but VPN fixed that, idk why 69yuedu doesn't work of bypass.)

Or Novel Link:
https://www.69yuedu.net/article/yjtxhehxsl.html
互联网时代，谁还做传统游戏？

1. It always stop at 1-40 chapters using VPN

2. Then have to switch VPN, or else it only crawl 1 chapter.

3. Switch VPN, Then tried to crawl at 41-83
   But it skip 41, and directly crawls at chapter 43

While page shows its crawling from 41, after finish crawling, it shows up to 83.
When i click in epub its shows chapter 43-83, not chapter 41-83

4. I using Firefox Developer Edition
   Since in store version of webtoepub on firefox don't work.
   (Not a developer), but use it due to it also has Temporary add on feature built In.

WebToEpub1.0.1.24.Firefox.zip for Firefox
WebToEpub1.0.1.24.Chrome.zip for Chrome

6. Note: i tried many VPN, but the only one time, i got past chapter 59,

Device:
Macbook Pro M3 14, Version:
macOS 14 | Sonoma | 14.7.2 (23H311)

[gamebeaker]

@steve30990 i guess you have to delete the cookies saved in the browser if you switch to vpn. If you only use the vpn with the same cookies the website still knows you are the same person.
The vpn could make it worse if multiple people that are using this vpn are accessing this site.
I was able to download all 295 Chapter with these options (in Chrome/ WebToEpub 1.0.1.25):

[dteviot]

@steve30990

Just in case it wasn't clear from gamebeaker, you're probably running into problems with WebToEpub fetching chapters too quickly and triggering the site's anti-scrapping code. So, you need to

1. Clear your cookies
2. Call in via a different VPN (or wait a day or two)
3. Reduce WebToEpub's fetch rate. (See screenshot in gamebeaker's response above.)