Subquadratic Ongororo paQuasa: LLM Yokutanga Ine Kukura Kuri Pasi Pechiyero Chesikweya

#Quasa #QUA #subquadratic

Subquadratic ikambani iri pamberi pakutsvakurudza nezveAI nekuvaka zvivako zvayo, iri kugadzira rudzi rutsva rwemamodheru makuru emutauro. Modheru yavo huru, SubQ, ndiyo LLM yokutanga yakavakirwa pahurongwa hunongonangisa kutarisisa kune zvimwe zvikamu zvemashoko, hune kukura kuri pasi pechiyero chesikweya zvizere. Kusiyana nemaTransformers akajairika, ayo anoda simba rekuverenga rinokura nechiyero chesikweya yekureba kwemashoko aanoshandisa ($O(n^2)$), simba rinodiwa neSubQ rinokura zvakaringana nekureba kwemashoko acho ($O(n)$), zvichiita kuti kufunga uchishandisa mamiriyoni akawanda ema token kugoneke pasina kupambadza zviwanikwa.

SubQ yakanyatsogadzirirwa mabasa anoda kushandisa mashoko akawanda panguva imwe chete. Inogona kugadzirisa dura rose rekodhi, nhoroondo refu dzezviito zvevamiririri veAI, zvikumbiro zvekubatanidza shanduko zvekumwedzi yakawanda, uye mamiriro anochengetwa nguva dzose, zvose zviri mumurayiro mumwe chete pasina kuderera kwemhando yezvinobuda. Parizvino modheru yacho inokwanisa kushandisa mashoko anosvika 12 million tokens panguva imwe chete uye inoita zvakanaka pamiyedzo yekutsvaga mashoko pakati pemashoko akawanda pamwe neyekufunga kwakasiyana-siyana. Pamashoko ane 1 million tokens, SubQ inoda simba rekuverenga rakaderera zvikuru (zvinonzi rakaderera ka64.5× pane kutarisisa zvikamu zvose) uye inomhanya nokukurumidza zvikuru kupfuura nzira dzakajairika dzekutarisisa mashoko.

Kambani yacho inotarisa pamusimboti wokuti “kushandisa zviwanikwa zvakanaka ndiko kungwara.” Nokugadzirisa chipingamupinyi chekukura kwechinodiwa nechiyero chesikweya pahwaro hwehurongwa hwemodheru, Subquadratic ine chinangwa chokugonesa chizvarwa chitsva chevamiririri veAI vanogona kushanda zvechokwadi nemashoko akawanda, panzvimbo pokuvimba nekuacheka kuita zvikamu, kuapfupisa, kana kushandisa nzira dzokunzvenga nadzo zvinetso zvekutsvaga mashoko.

Subquadratic yakanyatsokodzera vagadziri vezvirongwa, vamiririri veAI vanonyora kodhi, zvikwata zvemakambani, nevatsvakurudzi vanoda kufunga kwakavimbika vachishandisa mashoko akawanda — kunyanya pakuongorora kodhi yose yepurojekiti, kuvamiririri vanoshanda kwenguva refu, uye pamabasa akaoma anosanganisira magwaro akawanda.

Zvinonyanya kukosha

• Hurongwa hweLLM hwokutanga hune kukura kuri pasi pechiyero chesikweya zvizere (hunokura zvakaringana nekureba kwemashoko).
• Kushandisa mashoko ane mamiriyoni akawanda ema token panguva imwe chete (kusvika 12M tokens).
• Kugona zvikuru kutsvaga mashoko pakati pemashoko akawanda pasina kupambadza zviwanikwa.
• Yakagadzirirwa kushandisa matura akazara ekodhi uye kuchengetedza mamiriro emumiririri nguva dzose.
• API inowirirana neOpenAI uye kubatanidzwa nevamiririri vanonyora kodhi.

• Zvingada kufungwa nezvazvo

• Sezvo iri modheru iri pamberi pakutsvakurudza, hurongwa hwakakura hwezvishandiso zvainoshanda nazvo nemhando dzayo dzine hukuru hwakasiyana zvichiri kuwedzerwa.
• Inonyanya kukodzera mabasa anoshandisa mashoko akawanda pane hurukuro dzakajairika dzine mirayiro mipfupi.

Mutongo wezvose: nyeredzi 4.7/5

Subquadratic iri kugadzirisa chimwe chezvipingamupinyi zvikuru zvemaLLM aripo iye zvino — mutengo wekutarisisa mashoko unokura nechiyero chesikweya. Nokuita kuti kufunga uchishandisa mamiriyoni akawanda ema token kugoneke pachishandiswa simba rekuverenga rakaderera zvikuru, SubQ idanho rinokosha rinofambisira mberi hurongwa hwevamiririri veAI nezvirongwa zvinoshandisa mashoko akawanda.

Wanawo mubayiro weQUA nokunyora ongororo paQuasa.io!

Tanga: https://quasa.io/projects/subquadratic