Compile and Export all the HASS Info in an LLM-Optimized Way

[AnshulJ999]

Hi, so I regularly use AI like Claude, Gemini, Perplexity, etc., to help me customize my HASS setup.

My main challenge till now has been keeping it updated with specific setup: all the devices, entities, configs, dashboards, etc. I have to manually copy-paste and compile a lot of YAML and upload that for the AI to understand.

Since this integration seems to be getting all this data already and sending it to the LLMs for the automation suggestions; I was wondering if there was a way we could export this data for our personal use?

It would be much easier to use this data with our custom prompts for additional use cases.

Thanks.

[ITSpecialist111]

Hey 👋 yes I've been thinking the same. I've been working on so.e dashboards to do exactly that. The idea being you can choose from a list of automation ideas, and it will show the automation yaml, script yaml and devices required, all in one view.

It takes a lot of time to sit down and build these integrations, but I might need to create an addon for Hass to facilitate this.

This integration looks directly at the entities database, so easier at the moment to pick up and suggest new entities to automations, rather than look at everything (which is what I want to do btw).

I need to design a process to do the Big picture element, possibility through RAG process.

Thoughts?

[AnshulJ999]

I appreciate the quick response and all the work that goes into this!

My two cents:

1. Looking at everything from the HASS setup would definitely be the best; if we can give the AI a holistic contextual understanding of the whole setup, I think it should perform even better.

For example, this integration keeps recommending me a 'Welcome Home Lights' automation and 'Add On Updates'; maybe the context is making it focus too much on these aspects.

2. I think RAG is no longer the best solution considering we also have big context models like Gemini and Claude, which could potentially ingest the entire HASS config at once. And Flash/Haiku are considerably cheap.

But RAG is probably great as a cost-effective solution: maybe some tests by using a full context approach vs RAG can show more insights.

3. Since you mentioned an add-on as well, I can imagine a kind of chat interface right inside HA where we could talk to any AI model (even local ones) about our HASS setup, and it would help create the relevant YAML.

A step further, it may even be able to directly edit the files, like Cursor does. But this is probably a bit complex and a lot of hard work haha.

Exciting times ahead :D

[ITSpecialist111]

Update: Had some time this morning to create 1.2.0 in Alpha for testing, although I've added more access to the Home Assistant API's ;)
This error originated from a custom integration.

Logger: custom_components.ai_automation_suggester.coordinator
Source: custom_components/ai_automation_suggester/coordinator.py:412
integration: AI Automation Suggester (documentation, issues)
First occurred: 11:04:37 (2 occurrences)
Last logged: 11:05:59

Anthropic API error: {"type":"error","error":{"type":"invalid_request_error","message":"prompt is too long: 215072 tokens > 199999 maximum"}}

I've integrated additional elements from the Home Assistant API that could be useful context for the integration:

(1) Accessing State and the Event Bus
(2) Interacting with Services
(3) Device and Entity Registries
(4) Area and Scene Registries

What’s Changed?

coordinator.py:
Now loads the Entity, Device, and Area registries to gather richer context for each entity.
Adds area and device information into the prompt so the AI has more context for suggestions.
After receiving suggestions, we call a built-in service (logbook.log) to record that new suggestions were generated (demonstrating service interaction).
init.py:
Sets up an event listener (hass.bus.async_listen) on the event bus for a custom event ("ai_automation_suggester_update") to trigger suggestions manually. This shows how to leverage the event bus to react to external stimuli.

[ITSpecialist111]

I appreciate the quick response and all the work that goes into this!

My two cents:

1. Looking at everything from the HASS setup would definitely be the best; if we can give the AI a holistic contextual understanding of the whole setup, I think it should perform even better.

For example, this integration keeps recommending me a 'Welcome Home Lights' automation and 'Add On Updates'; maybe the context is making it focus too much on these aspects.

2. I think RAG is no longer the best solution considering we also have big context models like Gemini and Claude, which could potentially ingest the entire HASS config at once. And Flash/Haiku are considerably cheap.

But RAG is probably great as a cost-effective solution: maybe some tests by using a full context approach vs RAG can show more insights.

3. Since you mentioned an add-on as well, I can imagine a kind of chat interface right inside HA where we could talk to any AI model (even local ones) about our HASS setup, and it would help create the relevant YAML.

A step further, it may even be able to directly edit the files, like Cursor does. But this is probably a bit complex and a lot of hard work haha.

Exciting times ahead :D

Hey - Please can you try V1.2.0 when you have a moment ;)