Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

世田谷総合運動公園の当日利用状況を取得してJSON形式でR2に保存するバッチを作成する #52

Open
1 of 3 tasks
subaru-hello opened this issue Nov 13, 2024 · 1 comment
Open
1 of 3 tasks

世田谷総合運動公園の当日利用状況を取得してJSON形式でR2に保存するバッチを作成する #52

subaru-hello opened this issue Nov 13, 2024 · 1 comment
Assignees
@subaru-hello

Comments

@subaru-hello
Copy link
Owner

subaru-hello commented Nov 13, 2024 •
edited
Loading

仕様

  • 競技場DBからオブジェクトストレージを参照したら、当日の利用可能時間を取得できる
  • Json形式。必要なカラムは以下の通り

方法

  • サーバー：Cloud RunかAWS Lambda with EventBridgeで実装する。 firebase functionsがpythonランタイムをリリースしたからそれを使う
    runtime: python→scrapingに適しているため。
  • ストレージ：S3かR2にJson形式で保存
  • Cron：毎日7時50(毎日7時30~7時40に更新されている。)
    Json名: setagaya_sougou_undou_kouen-YYYYMMDD.json

サイトURL: https://www.se-sports.or.jp/facility/sougou/
当日利用状況：https://www.se-sports.or.jp/about/news/20241113-18032

ページの構造

  • サイトTOP
image DOM image
  • 当日利用状況
image

DOM
image
@subaru-hello subaru-hello self-assigned this Nov 13, 2024
This was referenced Nov 14, 2024
Merged
Merged
Merged
subaru-hello added a commit that referenced this issue Nov 22, 2024
@subaru-hello
Merge pull request #75 from subaru-hello/#52/feat-scraping
09d83a3 
世田谷総合運動場の個人利用状況を取得するfunctionの土台を作成
@subaru-hello subaru-hello mentioned this issue Nov 23, 2024
Merged
subaru-hello added a commit that referenced this issue Nov 23, 2024
@subaru-hello
Merge pull request #77 from subaru-hello/#52/scrape-today-kojin-avail…
b5cdae7 
…ability

本日（MM月DD日7時30分現在）総合運動場　個人開放のおしらせ を取得する
@subaru-hello
Copy link
Owner Author

cron化
https://firebase.google.com/docs/functions/schedule-functions?gen=2nd#python

@subaru-hello subaru-hello mentioned this issue Nov 24, 2024
Merged
subaru-hello added a commit that referenced this issue Nov 24, 2024
@subaru-hello
Merge pull request #82 from subaru-hello/#52/schedulernize
7308176 
feat: cron化させた
This was referenced Nov 24, 2024
Draft
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@subaru-hello subaru-hello

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@subaru-hello