Finally writing up how to get started using Claude/Codex to automate your home entirely locally - you'll be amazed at what you can do with them and @home_assistant https://x.com/i/article/2115518077890940928
Getting Started With Home Automation & Coding Agents
posted on my blog as well
I’m a big home automation nerd and coding LLMs have only accelerated it as the power they give to configure and program your home are unbelievable. I’m not talking about using an LLM to run things day to day, but rather just use them to create the deterministic routines that make everything run so you never touch a light switch or thermostat again.
My Approach
There’s a few principles I have in designing a smart home
1. It needs to fail gracefully - if the central home automation server stops working that shouldn’t break basic functionality like lights
1. I want local control for everything, no dependence on cloud servers
1. It’s called Home Automation for a reason, opening an app on your phone isn’t automation, things should just happen automatically based on sensors
1. If something can’t happen automatically, there should be a physical button that follows well understood conventions so people can use it easily.
1. If a button isn’t available, it should be doable via voice control. Only the rarest of instances should require a phone app.
My Stack
For someone looking for local control, the gold standard is Home Assistant. The thing that makes HA truly powerful though is the open source community behind it - they have reverse engineered integrations with almost every device and service you can think of, as well as built incredible plugins and customizations for it. All available for free. This is what makes the vision of local control and a single hub truly practical in a world that is normally dominated by terrible phone apps and vendor lock in.
HA lets you piece together a totally cohesive setup using best of breed devices from any manufacturer. I can use the best lights, the best blinds, the best sensors, regardless of who makes them or what protocol they use. And it can even act as a Homekit Bridge, so all my devices show up accessible to Siri for voice control, even if they aren’t natively compatible with Homekit.
The other key here is the “Unofficial and Awesome HA MCP”, which allows Claude and Codex to handle 100% of the administration and configuration of a Home Assistant instance. I started with HA before LLMs were a thing and so I can do plenty in there on my own, but the power of Claude to handle the grunt work, analyze patterns, create automations is unreal. Start with this from day one. It also makes debugging much easier as you can just tell your agent what seems off and it will go digging.
Automation Gone Wild
So what does this full automation look like? First I have several different house modes I defined of “Home”, “Away”, “Sleep”, and “Out of Town”. Switching between Home and Away happens automatically based on location sensors on our phones (again its all local so no privacy concerns) that know how many people are home at any given time. When we all leave, it switches to “Away” but you can define your own criteria. Other devices and sensors can feed other data in, gathering whatever datapoints you need for the scripting you want to accomplish.
Lighting
For lighting, we also capture sensor data such as from lux and motion sensors in the house and the combination of all these feed into a package someone built called Adaptive Lighting to turn the lights on/off and to the proper levels at different times of the day, it even does color shifting if you have full color bulbs (like we have in the bedrooms). End result, I basically never touch light switches in the main part of the house.
Shades
Maximal natural light is also a big priority for us, but so is not getting hit in the face with glare from the bright sun up here in Montana, so we have motorized shades in various parts of the house (check out SmartWings for very affordable priced ones). A package called Adaptive Cover Pro does a similar think as Adaptive Lighting but for shades: use your LLM to work out the math on your windows and what angles/heights in your house you want to protect from the sun and it does all the math over the course of the year to position the shades based on where the sun is in the sky, including overrides based on weather, cloud cover, temperature, occupancy etc.
For instance, we have two shades that really only need to be closed in the early mornings for about 6 weeks each, twice a year based on where the sun hits (and at various times over those 6 weeks). Adaptive Cover handles this so that they just stay open the rest of the time.
HVAC
For HVAC, we have two different systems (hydronic baseboard and minisplits) that handle heating and cooling for the house plus our temperatures can also range by 40 degrees Fahrenheit over the course of a single day. It’s complex. And so while there are packages out there meant to do smart control of home HVAC, none did quite what I wanted. However I was able to simple point Claude and Codex at the temperature history for the house, plus the sensors we have for current weather and forecast, describe how I want the climate control to work, and it built out a series of interconnected automations that automatically handle all of the climate without me having to touch a thermostat.
Obviously this requires some iteration (as do the other things) but that’s the great part - have your agent come up with an initial setup, use it for a few days, then have it review and discuss the data with you, make changes and repeat. After a couple of weeks, you should be locked in.
Random Other Things
There’s 1,000 other small automations and hacks I’ve built as well, as some examples:
- Our tractor with a snowblower lives in the haybarn during the winters, it’s engine block heater is hooked up to a smart plug that can automatically turn on at 4am to pre-heat the engine on days where it snowed overnight.
- When the camera on my desktop computer turns on (because I’m joining a video call), smart plugs automatically turn on the video lights around my computer and turn off the other lighting
- All of the automations for my office tv are also controlled by HA
- Alerts from our cameras when people pull into the driveway or are at the front door
- Buttons in the entry way to open/close the garage door at our detached garage (plus automations to automatically notify and close them if they get left open)
- many, many more
Building Your Own Hardware
If you really want to go down a rabbit hole, you can also start building your own custom hardware based with ESPHome, an open source framework for ESP32 microcontrollers.
For instance, when we built a sauna, I wanted to stick to my local control for it and not have another app, so I built by own sauna controller using ESP32s and a heavy duty relay that provides all the functionality of a commercial controller (and more!) at a fraction of the price, integrated into my existing home, and with no crappy app.
A Warning
I’ve listed a lot you can do with HA here and it’s truly incredible, the only limit really is your imagination and what you can do with an LLM. But I will caveat that this is a system for people who like to tinker. Things will occasionally break, updates will come, vendors will push firmware updates, etc. So you have to want to do this.
But damn, it’s so much fun.