A Different Leveraging of AI
Here's the thing: AI (or really, just a LLM), can do a lot. Having seen results (though not using AI myself), and the overall layout/architecture of an LLM here's nano-gpt (a lot there), and the hardware that makes that work – there's some common-ish sense as to what to do with this (technology). Potentially without supercomputers or enormous data centers.
1) For one, an open and contributable (not a word) “base” LLM. One that can be copied/replicated to as many repositories as one wishes. This skeletal LLM could then be built upon, changed, improved, streamlined, or concentrated to be “great” (effective) at many things, or made into a sort of ASIC (application-specific integrated circuit) of large scale computing. Sort of a “kernel”, of sorts, of AI.
2) Using known/public information to be implemented within this LLM. Examples: information in the dictionary, mathematics, from simple addition to Chaos Math, all programming languages, and effectively any large wealth of genuinely useful information that exists. Seems a lot, but storage is cheap ;)
3) Data use and knowledge pools: just because Github/Microsoft pillage the repositories on Github, use (with or without permission) a developers code (let's say Ruby) in order to answer specific queries one may ask of their AI service, doesn't mean that the manuals and documentation of Ruby (a programming language), aren't readily available to those that want to access it (in this case AI – or anyone or any thing). A piece of software (be it a LLM or other) that is trying to answer a question, should be able to answer it by “formulating” the thing being asked of it, and then provide a response. A (more tedious) “step-by-step/show your work” method of developing a LLM (and a more conscientious one) offers more benefit longterm and short term instead of AI sifting a repository real fast and cherry-picking nuanced answers. Investing and continually developing this approach (a LLM focused on formulation), would be much more effective and beneficial to Computer Science, and to people, than a “search>grab>display” approach (which is basically all AI does now).
4) Slimmer investment, lighter computing, less storage (costs): With the open and documented information in the world, and online, readily available and free/open knowledge (not data and not personal data – which is rarely procured with consent), and training, building, and refining what a LLM does with this open/free information, ensures not just accurate results, but can be refined, in time, to run on lighter hardware, and more refined software. If “XYZ LLM needs XYZ hardware to offers accurate results for XYZ use case”, then the element of predictability and usefulness becomes more apparent for this technology.
So,
1) an open, and freely contributable (again, not a word) LLM 2) Using known, open, public, free knowledge – not data from someone's repositories or personal information 3) A formulation method of improving/developing the LLM – sort of a large scale calculator, instead of a large scale search engine 4) Lowering the demands of CPU and storage requirements in order for this LLM to work. And with that, having a more useful, accessible, and predictable piece of Computer Science (re-inventing the wheel doesn't have to be the priority – just make something that's useful and reliable).
Part of the AI Notes series. Previous entries here