Commit Graph

344 Commits

Author SHA1 Message Date
7768a202b2
fix: failed to serialize bigint correctly 2025-03-23 20:23:00 +08:00
8652ac8fb7
fix: bugs of snapshot scheduling 2025-03-23 19:45:52 +08:00
18fc9752bb
fix: several bugs of snapshot scheduling 2025-03-23 18:46:25 +08:00
2c12310e8c
feat: snapshot based on persistent schedule 2025-03-23 17:44:59 +08:00
b201bfd64d
fix: add type assertions to suppress errors 2025-03-22 23:49:48 +08:00
e38dc96275
update: insertion of snapshot schedule 2025-03-22 20:51:28 +08:00
9f9ef800d1
merge: hotfix into ref/snapshot 2025-03-22 01:48:03 +08:00
e5534cda24
fix: incorrect filter condition that causes empty tags 2025-03-22 00:58:36 +08:00
559c63b434
update: more beautiful time interval formatting 2025-03-22 00:42:37 +08:00
1895d601d9
update: dynamic delay factor for snapshotMilestoneVideo 2025-03-22 00:40:00 +08:00
fabb77d98d
fix: inefficient SQL query for getting songs close to milestone 2025-03-22 00:28:47 +08:00
8158ce10c0
fix: inserting videos into songs table regardless of classified label 2025-03-21 21:06:01 +08:00
00b52c01f7
fix: unexpected column bvid when inserting to songs table 2025-03-21 20:51:34 +08:00
2e8ed7ce70
add: ETA estimation for short-term snapshot 2025-03-20 01:57:33 +08:00
ba6b8bd5b3
temp: try to modify some features for the pred model 2025-03-20 00:53:59 +08:00
cd8aa826e1
fix: prevent videos from being crawled for too long 2025-03-17 00:33:28 +08:00
b07d0c18f9
update: preparation for snapshotSchedule 2025-03-17 00:25:31 +08:00
2ed909268e
add: a short-term feature in pred model 2025-03-16 15:10:32 +08:00
a9ac8de547
fix: unhandled timezone mismatch when inserting to database 2025-03-16 14:23:11 +08:00
0ff1c78dcc
fix: incorrect timestamp unit when inserting to database 2025-03-16 14:00:49 +08:00
a6c8fd7f3f
ref: code structure related to AI 2025-03-16 01:23:10 +08:00
35b84787ad
doc: GitBook - No subject 2025-03-15 13:42:19 +00:00
7104a95af9
ref: rename table all_data, bili_user to bilibili_metadata, bilibili_user 2025-03-15 21:27:19 +08:00
5af2236109
temp: remove the scheduleSnapshotTick job 2025-03-15 21:25:26 +08:00
93bdddc21e
fix: unexpectedly commented code to upsert jobs 2025-03-15 16:51:07 +08:00
afffbd8ecb
merge: branch 'lab/pred' 2025-03-15 16:47:34 +08:00
60be2274e4
improve: use custom loss for the model 2025-03-15 16:47:23 +08:00
bf919da1ea
update: the features for the model 2025-03-15 05:07:49 +08:00
a6211782cb
update: more accurate short-term prediction 2025-03-15 04:19:04 +08:00
f0148ec444
add: the 'term' param in Dataset class 2025-03-15 02:56:28 +08:00
23e5d6a8c9
add: more features 2025-03-13 23:32:13 +08:00
1de8d85d2b
update: the stable pred model 2025-03-13 21:07:17 +08:00
296e4ef4d6
add: the pred model prototype 2025-03-12 03:51:39 +08:00
e12275dbd4
fix: zero division error 2025-03-11 23:50:21 +08:00
bce4161501
improve: better eta prediction 2025-03-11 23:40:15 +08:00
f177d96817
add: zed config 2025-03-10 01:19:05 +08:00
09f872b11a
improve: handling of invisible videos in snapshot 2025-03-09 23:01:10 +08:00
9a85da0532
update: handling for deleted videos 2025-03-09 22:57:26 +08:00
81b95c9569
update: minimum interval for snapshotMilestoneVideo 2025-03-09 22:52:12 +08:00
a138f6d572
add: auto-removal of jobs, aliyun-fc auto retry 2025-03-09 22:34:23 +08:00
d5a317ea34
update: save both stdout and stderr for aliyunfc error handling 2025-03-09 22:16:54 +08:00
2c503df172
add: raw output dump in aliyun-fc error handling 2025-03-09 22:11:46 +08:00
8721089e7c
update: proxies that can be used to snapshot 2025-03-09 22:02:16 +08:00
7ac2d2c217
update: completed snapshot for videos close to milestone 2025-03-09 19:36:19 +08:00
5cc2afeb0e
add: snapshot feat for new videos and songs near milestone 2025-03-09 05:35:20 +08:00
fa414e89ce
add: insert labelled songs into songs table 2025-03-08 00:55:29 +08:00
98e063e86c
add: a simple tdd spider 2025-03-06 23:37:13 +08:00
2a2e65804f
merge: branch 'feat/user-info' to main 2025-03-05 00:24:18 +08:00
da47c0ab66
update: alicloud-fc support in NetScheduler 2025-03-05 00:23:16 +08:00
3842c63ad1
add: filter model V3.17 with deployment 2025-03-03 02:09:45 +08:00