sparkastML

This repository houses the machine learning components for the sparkast project.

The primary objective of this project is to enhance the search functionality of sparkast, allowing users to receive real-time answers as they type their queries.

Intention Classification

The model located in the /intention-classify directory is designed to categorize user queries into predefined classes.

We utilize a Convolutional Neural Network (CNN) architecture in conjunction with an Energy-based Model for open-set recognition.

This model is optimized to be lightweight, ensuring it can run on a wide range of devices, including within the browser environment.

For a detailed explanation of how it works, you can refer to this blog post.

Translation

Language barriers are one of the biggest obstacles to communication between civilizations. In modern times, with the development of computer science and artificial intelligence, machine translation is bridging this barrier and building a Tower of Babel.

Unfortunately, many machine translations are owned by commercial companies, which seriously hinders the development of freedom and innovation.

Therefore, sparkastML is on the road to challenge commercial machine translation. We decided to tackle the translation between Chinese and English first. These are two languages ​​with a long history and a large number of users. Their writing methods and expression habits are very different, which brings challenges to the project.

For more details, you can view this page.

Dataset

To support the development of Libre Intelligence, we have made a series of datasets publicly available. You can access them here.