|
|
9ba1a0c6f0
|
update: collection of metrics of queues
|
2025-10-30 02:34:01 +08:00 |
|
|
|
125a30f0df
|
add: metrics for queue
|
2025-10-30 01:46:34 +08:00 |
|
|
|
154c075aaf
|
add: metrics monitoring
|
2025-10-29 23:56:23 +08:00 |
|
|
|
937482e50b
|
fix: currentSnapshot may be null in takeBulkSnapshotForVideosWorker
|
2025-10-26 23:39:53 +08:00 |
|
|
|
92dccbab9d
|
ref: full deployment config for crawler based on PM2
|
2025-10-26 22:38:22 +08:00 |
|
|
|
1a79e679bb
|
ref: use bun's redis in crawler, deployment via PM2
|
2025-10-26 17:26:11 +08:00 |
|
|
|
57992069ec
|
update: better logging
|
2025-10-21 22:50:38 +08:00 |
|
|
|
9770a487bc
|
add: logging for debugging
|
2025-10-21 22:25:57 +08:00 |
|
|
|
b25a2992e1
|
update: missing service & codePath info in some logging
|
2025-10-20 05:36:38 +08:00 |
|
|
|
39ae323269
|
fix: incorrect target for calculating eta
|
2025-10-08 04:43:53 +08:00 |
|
|
|
14be6d51b2
|
add: insert eta info into database
|
2025-10-08 04:35:47 +08:00 |
|
|
|
87d9066fa1
|
add: the server timing & prettifier plugin, update the archiveSnapshots logic
|
2025-10-08 04:03:00 +08:00 |
|
|
|
02a2a845da
|
ref: cleanup code
|
2025-10-06 17:04:16 +08:00 |
|
|
|
b40d24721c
|
add: milestone monitoring for videos close to N million views
|
2025-06-06 21:37:55 +08:00 |
|
|
|
8cf9395354
|
fix: don't adjust start time for schedule when no proxy is available
|
2025-06-06 17:11:44 +08:00 |
|
|
|
1e8d28e194
|
fix: incorrectly ignored type when collecting videos for archive snapshots
|
2025-06-06 16:52:27 +08:00 |
|
|
|
2c83b79881
|
update: termination condition to time-based in classifyVideosWorker
|
2025-05-31 12:23:01 +08:00 |
|
|
|
1a20d5afe0
|
update: schedule archive snapshots to next Saturday midnight
fix: no expire when acquiring lock for classifyVideos
ref: format
|
2025-05-31 12:13:56 +08:00 |
|
|
|
1ff71ab241
|
improve: code quality
|
2025-05-19 00:13:01 +08:00 |
|
|
|
79a37d927a
|
fix: forget to specify type when collecting videos without active schedules;
missing return when eta too long for milestone snapshot
|
2025-05-19 00:10:33 +08:00 |
|
|
|
b18b45078f
|
add: tokenID added to JWT in endpoint GET /captcha/:id/result
|
2025-05-17 02:15:05 +08:00 |
|
|
|
59f09ca5eb
|
ref: switch to bun
|
2025-05-05 01:53:33 +08:00 |
|
|
|
1a86831e90
|
fix: handle ALICLOUD_PROXY_ERR error in snapshotVideoWorker
|
2025-04-24 03:42:25 +08:00 |
|
|
|
2c47105913
|
fix: incorrectly delayed milestone schedule due to UPDATE
|
2025-04-17 02:18:25 +08:00 |
|
|
|
7689e687ff
|
ref: extract scheduleCleanup into individual file
improve: logic of scheduleCleanup
|
2025-04-15 03:50:03 +08:00 |
|
|
|
651eef0b9e
|
fix: incorrectly set snapshot status to 'completed' when bilibili returned other status code
|
2025-04-15 03:34:45 +08:00 |
|
|
|
68bd46fd8a
|
improve: logging text in snapshotVideo.ts
|
2025-04-14 05:39:27 +08:00 |
|
|
|
13ea8fec8b
|
improve: remove unused import in snapshotTick.ts
|
2025-04-14 05:37:06 +08:00 |
|
|
|
3d9e98c949
|
ref: extract snapshotVideoWorker into individual file
|
2025-04-14 05:36:07 +08:00 |
|
|
|
c7dd1cfc2e
|
improve: unused imports in snapshotTick.ts
|
2025-04-14 05:21:06 +08:00 |
|
|
|
e0a19499e1
|
ref: move some worker functions into individual files
|
2025-04-14 05:20:35 +08:00 |
|
|
|
0930bbe6f4
|
improve: remove unused import
|
2025-04-14 05:11:35 +08:00 |
|
|
|
054d28e796
|
merge: branch 'main' into ref/structure
|
2025-04-14 05:10:44 +08:00 |
|
|
|
6df6345ec1
|
update: status of schedule when no proxy available
|
2025-04-14 00:46:43 +08:00 |
|
|
|
21c918f1fa
|
fix: BitInt serialization in bulkSnapshotTickWorker
|
2025-04-14 00:32:06 +08:00 |
|
|
|
f1651fee30
|
fix: did not release lock for dispatchArchiveSnapshots when quitting
|
2025-04-14 00:28:45 +08:00 |
|
|
|
d0b7d93e5b
|
fix: incorrect logging text in archiveSnapshotsWorker
|
2025-04-14 00:26:59 +08:00 |
|
|
|
7a7c5cada9
|
feat: full snapshot archiving for all videos
|
2025-04-14 00:25:39 +08:00 |
|
|
|
9ef513eed7
|
update: add randomness in interval of re-scheduling when no proxy available
|
2025-04-13 23:55:43 +08:00 |
|
|
|
b080c51c3e
|
merge: branch 'main' into ref/structure
|
2025-04-13 18:38:34 +08:00 |
|
|
|
f4d08e944a
|
fix: expand the candidate range of milestone snapshot
|
2025-04-13 16:50:19 +08:00 |
|
|
|
a9582722f4
|
ref: wrap some worker functions with withDbConnection
|
2025-04-13 05:06:52 +08:00 |
|
|
|
4ee4d2ede9
|
ref: move some functions into separate files
|
2025-04-12 02:08:04 +08:00 |
|
|
|
fc90dad185
|
ref: move some common files to @core
|
2025-04-08 02:05:44 +08:00 |
|
|
|
b76d8e589c
|
merge: branch 'main' into ref/structure
|
2025-04-07 02:44:21 +08:00 |
|
|
|
69fb3604b1
|
fix: missing setSnapshotStatus in takeSnapshotForVideoWorker
|
2025-04-07 00:41:27 +08:00 |
|
|
|
d98e24b62f
|
fix: incorrect return from snapshotTickWorker
|
2025-04-07 00:36:44 +08:00 |
|
|
|
da1bea7f41
|
fix: unlimited addition of milestone tasks
|
2025-04-07 00:28:28 +08:00 |
|
|
|
a90747878e
|
fix: incorrect SQL ingetVideosNearMilestone
|
2025-04-06 16:21:46 +08:00 |
|
|
|
dd720b18fa
|
add: debug logging for collectMilestoneSnapshotsWorker
|
2025-04-06 16:19:05 +08:00 |
|
