Dear iodéOS developers. Is there, in the road map of your great OS, any possibility of implementation of any locally run AI model? In my limited knowledge of OS development, it usually depends mostly on used HW. And because iodéOS mostly runs on older HW, there is not a big chance that such a thing (local AI) could be implemented globally. So just want to peep into the window of future updates, if there is any chance of such possibility. Thanks for your great work.
Based on feedback from premium iodé users, we are working on our roadmap for new features, which will soon be announced, and the team is already working on a couple of these. Local AI is not one of the features we are currently considering. We do however, base our decisions primarily on the desires from the community, so if lots of community members ask for it, we will study the feasibility. AI is a loaded term and has many applications besides large language models, and the big tech companies who make LLMs do not help clear up this confusion, so it’d be helpful if you described what exactly it is you want an AI model to do on your phone.
3 Likes
OK. Firstly thanks for quick reply. Secondly, AI on device, at least in my point of view, is all about automatization and awareness of on device ongoings. For example possibility of speech driven prompts by a user, which is understand and done by phone itself, without any data leaving the device - setting reminders, appointments in the calendar, searching for a phrase in documents/apps residing in the phone etc. I am not sure if such a AI can and will infiltrate and corrupt privacy of the device, so just take it more of a question of feasibility than anything else.
2 Likes
@gc35sdd I think many of these things could be considered features of a “voice assistant” maybe? (setting alarms, calendar events, etc)?
There are various “speech to text” options such as from FUTO, I would expect such features to work a bit in collaboration with something like it through “Accessibility”, like can happen with “Text-to-Speech”. At least that is where I would see the development happening. Certainly it would take cpu cycles, the feature may not be feasible on under powered devices.
There are already many “awareness of device ongoings” that happen as I have noted in other threads (e.g. battery optimization, etc.).
1 Like
It would be nice to have some integration of self-hosted AI and the iodé phone. Something like those home servers like umbrel and start9 integration.
1 Like
@rik I am not developer of any SW, so not able to pinpoint exactly with right naming what I mean by AI on device application. For me it is more of a concept. AI is here and in many forms (as you have mentioned). Some useful, mostly just gimmicky stuff. But why not use something meaningful, what can make usage of mobile phones more fluent and easy. But I am sure the future will shape that way on its own. May be, that the voice interaction with any IO device is the easiest way how to imput all queries and get the results one wants. And for such a conversation you have to talk with something at least a bit aware and inteligent, aca in this situation with some sort of AI. The scary and malicious part of recent AI models on phones is its shameless spying, copying and stealing every bit of information it saw, heard, photographed etc. And it is done on purpose and because it lives not on the phone but realy in the cloud. If AI can be entrenched only in the phone itself, cut off by any outside communication, then it could be more of a help rather than enemy.
1 Like
if the AI is on your own server communcating on an encrypted way with your phone it’s fine. And your server can have a very powerful cpu like intel i9 last gen and 64gb of ram, that a phone will take decades to have, but also it consumes a lot of battery.
Check this: AI | Umbrel App Store
2 Likes
I am aware of such solution. And it is on my list (such a similarPC I have in my disposal). The not so friendly fact is its pretty enormous consumption of energy (it has to run 24 hrs a day), production of lot of heat and even it has a lot of quiet fans (5 in my case), it is still substantial noisy. So for normal usage like NAS with AI capabilities it is nonsense, if it resides in your residential area. For some minor AI tasks you can use normal NAS with stronger CPU, but right now, prices of RAM, which is necessity for AI tasks, is stupidly high (of course because of AI crazy all over the world).
you can run it on a rpi5, it’s cheaper and uses less energy.