Subquadratic kuri Quasa: Isuzuma rya LLM ya mbere ya Sub-Quadratic

#Quasa #QUA #subquadratic

Subquadratic ni ishirahamwe rikora ubushakashatsi n'ibikorwa remezo vya AI ku rugero rwo hejuru, ririko rirubaka umugwi mushasha w'imideli minini y'ururimi. Umudeli waryo nyamukuru, SubQ, ni LLM ya mbere yubakiye ku buryo bwa sparse-attention bwa sub-quadratic mu mice yabwo yose. Mu gihe Transformers zisanzwe zisaba ubushobozi bwo guharura bwiyongera hakurikijwe kwadarati y'uburebure bw'amakuru ahabwa umudeli ($O(n^2)$), ubusabwa na SubQ bwiyongera ku rugero rumwe n'ubwo burebure ($O(n)$), bigatuma kwiyumvira ku makuru arimwo tokens amamiliyoni menshi bishoboka kandi bigakoresha neza ubushobozi buhari.

SubQ yagenewe canecane ibikorwa bisaba amakuru maremare ahabwa umudeli. Irashobora gukorera ku bubiko bwose bwa code, amateka maremare y'ibikorwa vy'abakozi ba AI, pull requests zo mu mezi menshi, n'amakuru y'uko ibikorwa bigeze abikwa igihe kirekire, vyose muri prompt imwe ata kugabanuka kw'ubuziranenge. Ubu uwo mudeli ushigikira amakuru ahabwa umudeli ashika kuri tokens imiliyoni 12 kandi utanga umwimbu mwiza mu bipimo vyo kurondera amakuru mu nyandiko ndende no mu vyo kwiyumvira muri rusangi. Kuri tokens umiliyoni 1, SubQ isaba ubushobozi bwo guharura buke cane (bivugwa ko ari buke incuro 64.5× ugereranije na dense attention) kandi ikora ningoga cane kurusha uburyo busanzwe bwa attention.

Iryo shirahamwe ryisunga ingingo ivuga ko “gukoresha neza ubushobozi ari ubwenge.” Mu gutorera umuti ku rwego rw'inyubako y'umudeli intambamyi iterwa n'ukwiyongera kw'ibisabwa hakurikijwe kwadarati, Subquadratic ifise intumbero yo gutuma habaho iyaruka rishasha ry'abakozi ba AI bashobora vy'ukuri gukorana n'amakuru menshi bahabwa, aho kwisunga ukuyagaburamwo ibice, ukuyavuga mu ncamake, canke amayeri yo kuyarondera.

Subquadratic irabereye cane abahingura porogarama, abakozi ba AI bandika code, imigwi yo mu mashirahamwe, n'abashakashatsi bakeneye ukwiyumvira kwizigirwa ku makuru maremare — canecane mu gusuzuma code yose y'umugambi, mu gukoresha abakozi ba AI bakora igihe kirekire, no mu bikorwa bigoye bisaba inyandiko nyinshi.

Ibintu nyamukuru

• Inyubako ya mbere ya LLM ya sub-quadratic mu mice yayo yose (ubushobozi bwo guharura bukenewe bwiyongera ku rugero rumwe n'uburebure bw'amakuru).
• Amakuru ahabwa umudeli arimwo tokens amamiliyoni menshi (ashika kuri tokens 12M).
• Ubushobozi bukomeye bwo kurondera amakuru mu nyandiko ndende hakoreshejwe neza cane ubushobozi buhari.
• Yagenewe gukorera ku bubiko bwose bwa code no ku makuru y'uko ibikorwa vy'umukozi wa AI bigeze abikwa igihe kirekire.
• API ikorana na OpenAI n'uburyo bwo kuyihuza n'abakozi ba AI bandika code.

• Ibintu vyoshobora kwitonderwa

• Kubera ko ari umudeli w'ubushakashatsi uri ku rugero rwo hejuru, urusobe rwagutse rw'ibikoresho n'ibikorwa bijanye na wo be n'imideli y'ubunini butandukanye biracabandanya kwaguka.
• Urabereye cane ibikorwa bisaba amakuru maremare kurusha ibiganiro bisanzwe bishingiye kuri prompts ngufi.

Isuzuma rya nyuma muri rusangi: inyenyeri 4.7/5

Subquadratic iriko iratorera umuti imwe mu ntambamyi zihambaye cane za LLMs z'ubu — ubushobozi bwo guharura attention isaba bwiyongera hakurikijwe kwadarati y'uburebure bw'amakuru. Mu gutuma kwiyumvira ku makuru arimwo tokens amamiliyoni menshi bishoboka hakoreshejwe ubushobozi bwo guharura buke cane, SubQ iserukira intambwe ihambaye mu nyubako y'imideli ikoreshwa n'abakozi ba AI hamwe na porogarama zikoresha amakuru maremare.

Na wewe ronka agashimwe ka QUA mu gutanga isuzuma kuri Quasa.io!

Tangura: https://quasa.io/projects/subquadratic