Subquadratic pa Quasa: Ndemanga pa LLM Yoyamba Yokhala ndi Kapangidwe ka Sub-Quadratic

#Quasa #QUA #subquadratic

Subquadratic ndi kampani yochita kafukufuku wapamwamba wa AI ndi kumanga zomangamanga zake, yomwe ikupanga mtundu watsopano wa ma modeli akuluakulu a chinenero. Modeli yake yaikulu, SubQ, ndi LLM yoyamba yomangidwa pa kapangidwe ka sparse-attention komwe kuchuluka kwa mawerengedwe ake konse kumakula mosafika pa sikwea ya utali wa nkhani. Mosiyana ndi ma Transformer achikhalidwe omwe kuchuluka kwa mawerengedwe ake kumakula motsatira sikwea ya utali wa nkhani ($O(n^2)$), kuchuluka kwa mawerengedwe a SubQ kumakula molingana ndi utaliwo ($O(n)$), zomwe zimapangitsa kuti kusanthula pogwiritsa ntchito ma token mamiliyoni ambiri kutheke komanso kugwiritse ntchito bwino zipangizo zowerengera.

SubQ yapangidwa makamaka kuti igwire ntchito zokhudza nkhani zazitali. Imatha kusanthula nkhokwe zonse za khodi, mbiri zazitali za zochita za ma agent, zopempha zophatikiza khodi za miyezi ingapo, komanso mikhalidwe yosungidwa mosalekeza, zonse m'langizo limodzi popanda kutsitsa ubwino wa zotsatira. Panopa modeli imeneyi imathandiza kugwiritsa ntchito nkhani zokwana 12 million tokens ndipo imachita bwino pa mayeso opeza mfundo m'nkhani zazitali komanso pa mayeso a kulingalira ndi kusanthula zinthu zosiyanasiyana. Pa 1 million tokens, SubQ imafuna mphamvu zochepa kwambiri zowerengera (akuti zochepa ndi 64.5× poyerekeza ndi attention yogwira ntchito pa ma token onse) ndipo imagwira ntchito mofulumira kwambiri kuposa njira zachizolowezi za attention.

Kampaniyi imatsatira mfundo yakuti “kugwiritsa ntchito bwino zinthu ndi luntha.” Pothetsa vuto la kuchuluka kwa mawerengedwe motsatira sikwea kuyambira pa kapangidwe kake, Subquadratic ikufuna kuthandiza kuti pakhale m'badwo watsopano wa ma agent a AI omwe angagwiredi ntchito ndi nkhani zambiri m'malo modalira kuzigawa m'magawo ang'onoang'ono, kuzifupikitsa, kapena njira zongothandizira kupeza mfundo.

Subquadratic ndi yoyenera kwa opanga mapulogalamu, ma agent olemba khodi, magulu a m'mabizinesi, ndi ofufuza omwe amafunikira kulingalira ndi kusanthula kodalirika pogwiritsa ntchito nkhani zazitali — makamaka posanthula khodi yonse ya pulojekiti, pa ma agent ogwira ntchito kwa nthawi yaitali, komanso pa ntchito zovuta zokhudza zikalata zambiri.

Mfundo zazikulu

• Kapangidwe koyamba ka LLM komwe kuchuluka kwa mawerengedwe ake konse kumakula mosafika pa sikwea (kumakula molingana ndi utali wa nkhani).
• Nkhani yokhala ndi ma token mamiliyoni ambiri (mpaka 12M tokens).
• Kupeza bwino mfundo m'nkhani zazitali pogwiritsa ntchito bwino kwambiri zipangizo zowerengera.
• Yapangidwa kuti igwire ntchito ndi nkhokwe zonse za khodi komanso kusunga mkhalidwe wa agent mosalekeza.
• API yogwirizana ndi OpenAI komanso kulumikizana ndi ma agent olemba khodi.

• Zinthu zomwe mungafunike kuziganizira

• Popeza ndi modeli ya kafukufuku wapamwamba, mapulogalamu ndi zida zogwirizana nayo, komanso mitundu yake ya makulidwe osiyanasiyana, zikupitirizabe kuwonjezeka.
• Ndi yoyenera kwambiri pa ntchito zogwiritsa ntchito nkhani zazitali kuposa pa macheza wamba okhala ndi malangizo achidule.

Chigamulo chonse: nyenyezi 4.7/5

Subquadratic ikuthana ndi chimodzi mwa zopinga zofunika kwambiri za ma LLM amakono — mtengo wa mawerengedwe a attention womwe umakula motsatira sikwea. Pothandiza kuti kulingalira ndi kusanthula pogwiritsa ntchito ma token mamiliyoni ambiri kutheke ndi mphamvu zochepa kwambiri zowerengera, SubQ ikuimira kupita patsogolo kwakukulu kwa kapangidwe ka ma agent ndi mapulogalamu ogwiritsa ntchito nkhani zazitali.

Nanunso pezani mphotho ya QUA polemba ndemanga pa Quasa.io!

Yambani: https://quasa.io/projects/subquadratic