张瀚铎Zhang Handuo

DeepSpeed + Accelerate

.

🧠 Memory Definition in LLM Applications

The ultimate goal is to generate personalized models. The actual work involves reverse engineering, using data generated by the model to infer the model itself. The model and training data are equivalent; the model is a compression of the training data. To achieve the ultimate goal, the model must be capable of on-device real-time learning, requiring a breakthrough in the current training -> inference paradigm. Training and inference must be integrated, with two core challenges - real-time updates (incremental training) and de-averaging individual data. The mid-term goal is to layer individual data (long-term memory) on top of a foundational model (pre-trained LLM).

Seq2seq Model with Attention

Sequence-to-sequence models are deep learning models that have achieved a lot of success in tasks like machine translation, text summarization, and image captioning.

How to remotely edit your project without having to use VIM

Remotely editing your work when your server does not have public IP address and you don’t want to spend any money is not so easy. Maybe you can use Team viewer or Anydesk or even chrome remote desktop, but there are high latencies. Maybe you can use ngrok to remotely ssh to your server, you have to use vim and you are not familiar with it at all 😧. I tried to use rmate but it is not convinient to edit across different files in a folder.

You only look once (YOLO) -- (2)

YOLO has higher localization errors and the recall (measure how good to locate all objects) is lower, compared to SSD. YOLOv2 is the second version of the YOLO with the objective of improving the accuracy significantly while making it faster.

Pooling Layer in CNN (1)

Today I didn’t have the mood to continue my work on map merging of different cameras. So I read the paper from DeepMind of Learned Deformation Stability in Convolutional Neural Networks recommended by Wang Chen.

SVO相关问题

1. ORBSLAM vs SVO

   • SVO （单目）

     • 优点： 速度极快，100多帧，在低端计算机上也能达到实时性。追踪和建图两个线程，追踪线程和ptam或者orbslam很像，也是建立误差项，然后refine和BA，区别就是用的直接法的Image alignment而不是特征点几何位置信息。

     • 缺点：只是里程计，没有后端优化和回环检测，所以累计误差较大，而且一旦丢了就挂了，没法重定位; 而且在设计的时候针对的是俯视的无人机摄像头，对于平视的摄像机效果很差; 拥有直接法的所有缺点：怕光照变化，怕模糊，怕大运动

   svo还有一个不是缺点的缺点，它开源的代码有好多好多坑，作者很多私货故意没有写进开源的代码里面，所以实际用的时候有很多问题，需要自己根据情况改进。

   • OrbSLAM

     • 优点：支持单目，双目，RGBD，是一个完整的系统，包含了里程计，特征点建图BA，回环检测三个独立线程，在i7上大概15～20hz（跟输入图像大小以及参数设置有关），精确度在近年来属于比较高的了。综合能力最强。

     • 缺点：ORB的提取以及match耗时较大，过快的旋转可能会丢失。而且因为三个线程会给CPU带来较大负担，基本没办法再跑其他大型算法了。对于场景特征点丰富要求高，某些场景如果没什么特征可能就会失败或者不准确。

   能够跟orbslam pk的是比较新的DSO算法，SVO除了速度基本各个方面被吊打。

   Read more