Mega Data Centers? NO. Your iPhone Is an OpenAI-Compatible LLM Server Now

[AZJonnie]

4. This 'bridge' discussed is provided so your local (i.e. ones on your phone) development tools

can access the mini-LLM that's also running on your phone. It's not a bridge for the public to use your mini-LLM like a peer-to-peer network, as it sounds like you're thinking.

For example I develop with tools from an outfit called JetBrains on my PC. I can connect my Jetbrains tools to various LLM's to help with things like coding suggestions as I'm typing. One of those is Ollama, which I have running locally on my PC. Ollama exposes standard HTTP-based API's which let any of my local dev tools connect to it and be serviced by it.

The way the local LLM on the Apple phone works by default is more locked down, I wouldn't be able to connect my Jetbrains tools (if I had them on my phone) to a local instance of Apple's LLM, even though it's right there on the phone. This bridge they're talking about standardizes how the local Apple LLM can be connected to, such that my Jetbrains tools COULD connect to it and leverage it with things like code completion.

It's meant to be used for local connections, apps on your phone connecting to the LLM on your phone, using standard protocols. Not to turn your phone into mini-datacenter for the public to use.