|
|
e5534cda24
|
fix: incorrect filter condition that causes empty tags
|
2025-03-22 00:58:36 +08:00 |
|
|
|
559c63b434
|
update: more beautiful time interval formatting
|
2025-03-22 00:42:37 +08:00 |
|
|
|
1895d601d9
|
update: dynamic delay factor for snapshotMilestoneVideo
|
2025-03-22 00:40:00 +08:00 |
|
|
|
fabb77d98d
|
fix: inefficient SQL query for getting songs close to milestone
|
2025-03-22 00:28:47 +08:00 |
|
|
|
8158ce10c0
|
fix: inserting videos into songs table regardless of classified label
|
2025-03-21 21:06:01 +08:00 |
|
|
|
00b52c01f7
|
fix: unexpected column bvid when inserting to songs table
|
2025-03-21 20:51:34 +08:00 |
|
|
|
cd8aa826e1
|
fix: prevent videos from being crawled for too long
|
2025-03-17 00:33:28 +08:00 |
|
|
|
a9ac8de547
|
fix: unhandled timezone mismatch when inserting to database
|
2025-03-16 14:23:11 +08:00 |
|
|
|
0ff1c78dcc
|
fix: incorrect timestamp unit when inserting to database
|
2025-03-16 14:00:49 +08:00 |
|
|
|
7104a95af9
|
ref: rename table all_data, bili_user to bilibili_metadata, bilibili_user
|
2025-03-15 21:27:19 +08:00 |
|
|
|
93bdddc21e
|
fix: unexpectedly commented code to upsert jobs
|
2025-03-15 16:51:07 +08:00 |
|
|
|
e12275dbd4
|
fix: zero division error
|
2025-03-11 23:50:21 +08:00 |
|
|
|
bce4161501
|
improve: better eta prediction
|
2025-03-11 23:40:15 +08:00 |
|
|
|
09f872b11a
|
improve: handling of invisible videos in snapshot
|
2025-03-09 23:01:10 +08:00 |
|
|
|
9a85da0532
|
update: handling for deleted videos
|
2025-03-09 22:57:26 +08:00 |
|
|
|
81b95c9569
|
update: minimum interval for snapshotMilestoneVideo
|
2025-03-09 22:52:12 +08:00 |
|
|
|
a138f6d572
|
add: auto-removal of jobs, aliyun-fc auto retry
|
2025-03-09 22:34:23 +08:00 |
|
|
|
d5a317ea34
|
update: save both stdout and stderr for aliyunfc error handling
|
2025-03-09 22:16:54 +08:00 |
|
|
|
2c503df172
|
add: raw output dump in aliyun-fc error handling
|
2025-03-09 22:11:46 +08:00 |
|
|
|
8721089e7c
|
update: proxies that can be used to snapshot
|
2025-03-09 22:02:16 +08:00 |
|
|
|
7ac2d2c217
|
update: completed snapshot for videos close to milestone
|
2025-03-09 19:36:19 +08:00 |
|
|
|
5cc2afeb0e
|
add: snapshot feat for new videos and songs near milestone
|
2025-03-09 05:35:20 +08:00 |
|
|
|
fa414e89ce
|
add: insert labelled songs into songs table
|
2025-03-08 00:55:29 +08:00 |
|
|
|
2a2e65804f
|
merge: branch 'feat/user-info' to main
|
2025-03-05 00:24:18 +08:00 |
|
|
|
da47c0ab66
|
update: alicloud-fc support in NetScheduler
|
2025-03-05 00:23:16 +08:00 |
|
|
|
3842c63ad1
|
add: filter model V3.17 with deployment
|
2025-03-03 02:09:45 +08:00 |
|
|
|
748e2e2aaa
|
add: support for inference of filter model V3.13
|
2025-03-03 01:06:55 +08:00 |
|
|
|
47e47f2b12
|
add: func:getUnArchivedBiliUsers
|
2025-03-02 18:48:01 +08:00 |
|
|
|
1720ff332e
|
fix: undefined reference of userRow
|
2025-03-02 18:39:36 +08:00 |
|
|
|
012887d1d9
|
fix: missing uid in bili_user causes undefined reference
|
2025-03-02 18:17:43 +08:00 |
|
|
|
f7e71c22f6
|
fix: empty author_info passed in that causes filter model to crash
|
2025-03-02 03:02:27 +08:00 |
|
|
|
5a22564526
|
fix: incorrect limiter key in triggerLimiter
|
2025-03-02 03:00:28 +08:00 |
|
|
|
5b0e27465b
|
update: author_info in classification
|
2025-03-02 02:54:30 +08:00 |
|
|
|
ecb44b9cba
|
update: interval between 2 getVideoInfo, rate limit for this task
|
2025-03-02 02:45:48 +08:00 |
|
|
|
1838219f04
|
ref: separate fetching of aid list and video metadata
|
2025-03-02 02:38:50 +08:00 |
|
|
|
c67e3d8e36
|
fix: incorrect key in setProviderLimiter of Scheduler
|
2025-02-26 01:48:18 +08:00 |
|
|
|
7566722d04
|
update: support for provider-level limiter in NetScheduler
add: fn:getLatestVideoAids()
|
2025-02-26 01:40:01 +08:00 |
|
|
|
232585594a
|
add: provider in NetScheduler, missing await
|
2025-02-26 00:55:48 +08:00 |
|
|
|
15312f4078
|
update: filter model benchmark
|
2025-02-22 22:53:40 +08:00 |
|
|
|
73b96e869d
|
fix: move filter model init to an earlier stage
|
2025-02-22 22:29:30 +08:00 |
|
|
|
95fa08b517
|
update: increase db pool limit
|
2025-02-22 22:21:09 +08:00 |
|
|
|
d2f9f28608
|
update: limited db pool size to save memory
|
2025-02-22 22:06:51 +08:00 |
|
|
|
c23753aceb
|
update: add lock for classifyVideo
|
2025-02-22 22:00:34 +08:00 |
|
|
|
46191cfd56
|
fix: unintended local_files_only in loading the tokenizer
|
2025-02-22 21:48:17 +08:00 |
|
|
|
f70401846a
|
add: more logging for filter's workers
|
2025-02-22 21:43:09 +08:00 |
|
|
|
b14a43f63f
|
fix: feeding empty data to the filter model
|
2025-02-22 20:56:57 +08:00 |
|
|
|
cecc1c1d2c
|
add: classifyVideo & classifyVideos implementation
|
2025-02-22 19:57:52 +08:00 |
|
|
|
7946cb6e96
|
fix: try to remove timezone offseet in bisect
|
2025-02-14 03:13:32 +08:00 |
|
|
|
4b48357ab6
|
update: insert duration as metadata
|
2025-02-14 03:06:39 +08:00 |
|
|
|
f78f7fabdd
|
ref: remove data/ from git
update: inference code for filter model
|
2025-02-14 02:04:07 +08:00 |
|
