|
|
aea9e10d1a
|
update: bulk fetch
|
2025-03-27 02:33:54 +08:00 |
|
|
|
d44ba8a0ae
|
fix: incorrect offset calculation in adjustSnapshotTime
|
2025-03-27 02:13:33 +08:00 |
|
|
|
d6dd4d9334
|
fix: potential shifting for obtained window offset in last commit
|
2025-03-27 02:09:48 +08:00 |
|
|
|
d5c278ae06
|
improve: cache result for adjustSnapshotTime
|
2025-03-27 01:48:10 +08:00 |
|
|
|
92678066a7
|
update: forgot to plus one for iteration count
|
2025-03-26 23:37:12 +08:00 |
|
|
|
afee7f58bf
|
update: return when meeting non-0 video status code in snapshotVideo job
|
2025-03-26 23:25:07 +08:00 |
|
|
|
5450c17e13
|
update: reduce rate limit, no longer collect deleted videos for milestone monitoring
|
2025-03-26 23:13:53 +08:00 |
|
|
|
b286a9d7b1
|
update: use findSnapshotBefore instead of findClosetSnapshot in regular snapshot scheduling
|
2025-03-26 22:59:52 +08:00 |
|
|
|
cabb360a16
|
fix: missing entrance for schedule type 'new'
update: perf log text
|
2025-03-25 22:58:56 +08:00 |
|
|
|
17ded63758
|
update: perf monitor for adjustSnapshotTime
|
2025-03-25 22:38:57 +08:00 |
|
|
|
a178b7fc16
|
update: force schedule for new songs
|
2025-03-25 04:52:19 +08:00 |
|
|
|
6b7142a6d5
|
fix: new jobs may be scheduled before the current time
|
2025-03-24 23:43:54 +08:00 |
|
|
|
fa058b22fe
|
fix: job consumption rate too low, add outdated job cleanup
|
2025-03-24 23:36:01 +08:00 |
|
|
|
48b1130cba
|
feat: continuous monitoring of new songs
|
2025-03-24 04:53:43 +08:00 |
|
|
|
42db333d1a
|
temp: schedule for new songs
|
2025-03-24 04:40:10 +08:00 |
|
|
|
0a73d28623
|
fix: incorrect time base when adjusting snapshot time
|
2025-03-24 04:15:51 +08:00 |
|
|
|
584a1be9f9
|
test: debug for adjusting snapshot time
|
2025-03-24 04:08:34 +08:00 |
|
|
|
20731c0530
|
test: debug for init window
|
2025-03-24 04:05:22 +08:00 |
|
|
|
cb573e55d9
|
fix: incorrect offset when initializing
|
2025-03-24 04:01:44 +08:00 |
|
|
|
3028dc13c7
|
improve: performance when dispatching jobs
|
2025-03-24 02:35:55 +08:00 |
|
|
|
0be961e709
|
improve: target time finding
|
2025-03-23 23:31:16 +08:00 |
|
|
|
35d58be8fd
|
fix: may accessing a non-existent schedule
|
2025-03-23 21:42:05 +08:00 |
|
|
|
3a0dd26c68
|
fix: only 1 pending task at a time
|
2025-03-23 21:36:40 +08:00 |
|
|
|
9e764746fb
|
update: better task retrieval for snapshotTick
add: logging when snapshot the video
|
2025-03-23 20:49:41 +08:00 |
|
|
|
33c63fc29f
|
fix: NaN caused by not converting date string to timestamp
|
2025-03-23 20:41:13 +08:00 |
|
|
|
7768a202b2
|
fix: failed to serialize bigint correctly
|
2025-03-23 20:23:00 +08:00 |
|
|
|
8652ac8fb7
|
fix: bugs of snapshot scheduling
|
2025-03-23 19:45:52 +08:00 |
|
|
|
18fc9752bb
|
fix: several bugs of snapshot scheduling
|
2025-03-23 18:46:25 +08:00 |
|
|
|
2c12310e8c
|
feat: snapshot based on persistent schedule
|
2025-03-23 17:44:59 +08:00 |
|
|
|
e38dc96275
|
update: insertion of snapshot schedule
|
2025-03-22 20:51:28 +08:00 |
|
|
|
9f9ef800d1
|
merge: hotfix into ref/snapshot
|
2025-03-22 01:48:03 +08:00 |
|
|
|
fabb77d98d
|
fix: inefficient SQL query for getting songs close to milestone
|
2025-03-22 00:28:47 +08:00 |
|
|
|
2e8ed7ce70
|
add: ETA estimation for short-term snapshot
|
2025-03-20 01:57:33 +08:00 |
|
|
|
cd8aa826e1
|
fix: prevent videos from being crawled for too long
|
2025-03-17 00:33:28 +08:00 |
|
|
|
b07d0c18f9
|
update: preparation for snapshotSchedule
|
2025-03-17 00:25:31 +08:00 |
|
|
|
a6c8fd7f3f
|
ref: code structure related to AI
|
2025-03-16 01:23:10 +08:00 |
|
|
|
7104a95af9
|
ref: rename table all_data, bili_user to bilibili_metadata, bilibili_user
|
2025-03-15 21:27:19 +08:00 |
|
|
|
5af2236109
|
temp: remove the scheduleSnapshotTick job
|
2025-03-15 21:25:26 +08:00 |
|
|
|
e12275dbd4
|
fix: zero division error
|
2025-03-11 23:50:21 +08:00 |
|
|
|
bce4161501
|
improve: better eta prediction
|
2025-03-11 23:40:15 +08:00 |
|
|
|
7ac2d2c217
|
update: completed snapshot for videos close to milestone
|
2025-03-09 19:36:19 +08:00 |
|
|
|
5cc2afeb0e
|
add: snapshot feat for new videos and songs near milestone
|
2025-03-09 05:35:20 +08:00 |
|
|
|
fa414e89ce
|
add: insert labelled songs into songs table
|
2025-03-08 00:55:29 +08:00 |
|
|
|
47e47f2b12
|
add: func:getUnArchivedBiliUsers
|
2025-03-02 18:48:01 +08:00 |
|
|
|
1720ff332e
|
fix: undefined reference of userRow
|
2025-03-02 18:39:36 +08:00 |
|
|
|
012887d1d9
|
fix: missing uid in bili_user causes undefined reference
|
2025-03-02 18:17:43 +08:00 |
|
|
|
5b0e27465b
|
update: author_info in classification
|
2025-03-02 02:54:30 +08:00 |
|
|
|
1838219f04
|
ref: separate fetching of aid list and video metadata
|
2025-03-02 02:38:50 +08:00 |
|
|
|
95fa08b517
|
update: increase db pool limit
|
2025-02-22 22:21:09 +08:00 |
|
|
|
d2f9f28608
|
update: limited db pool size to save memory
|
2025-02-22 22:06:51 +08:00 |
|
