Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: kwu_de source not found #2893

Open
5 of 7 tasks
Jirka-Eh opened this issue Oct 21, 2024 · 25 comments
Open
5 of 7 tasks

[Bug]: kwu_de source not found #2893

Jirka-Eh opened this issue Oct 21, 2024 · 25 comments

Comments

@Jirka-Eh
Copy link

I Have A Problem With:

A specific source

What's Your Problem

2024-10-21 18:29:27.425 ERROR (SyncWorker_4) [custom_components.waste_collection_schedule.waste_collection_schedule.source_shell] source not found: kwu_de

Source (if relevant)

kwu_de

Logs

2024-10-21 18:29:27.425 ERROR (SyncWorker_4) [custom_components.waste_collection_schedule.waste_collection_schedule.source_shell] source not found: kwu_de

Relevant Configuration

No response

Checklist Source Error

  • Use the example parameters for your source (often available in the documentation) (don't forget to restart Home Assistant after changing the configuration)
  • Checked that the website of your service provider is still working
  • Tested my attributes on the service provider website (if possible)
  • I have tested with the latest version of the integration (master) (for HACS in the 3 dot menu of the integration click on "Redownload" and choose master as version)

Checklist Sensor Error

  • Checked in the Home Assistant Calendar tab if the event names match the types names (if types argument is used)

Required

  • I have searched past (closed AND opened) issues to see if this bug has already been reported, and it hasn't been.
  • I understand that people give their precious time for free, and thus I've done my very best to make this problem as easy as possible to investigate.
@5ila5
Copy link
Collaborator

5ila5 commented Oct 22, 2024

This should not happen and may indicate a broken install, maybe try removing and re-installing WCS (don't forget to reboot HA)

@Jirka-Eh
Copy link
Author

Jirka-Eh commented Oct 22, 2024

I removed WCS, restarted HA, deleted old entities, restarted HA again.
Than I reinstalled WCS and restarted HA.
Add new integration -> WCS -> configure it.
Country: Germany
Source: kwu_de => Unknown error occurred

Logger: custom_components.waste_collection_schedule.waste_collection_schedule.source_shell
Quelle: custom_components/waste_collection_schedule/waste_collection_schedule/source_shell.py:221
Integration: Waste Collection Schedule (Dokumentation)
Erstmals aufgetreten: 18:29:17 (1 Vorkommnisse)
Zuletzt protokolliert: 18:29:17

source not found: kwu_de

Core
2024.10.3
Supervisor
2024.10.2
Operating System
13.2

Maybe the URLs at https://www.kwu-entsorgung.de/ have been changed?
WCS worked perfectly until last week!

@5ila5
Copy link
Collaborator

5ila5 commented Oct 22, 2024

I cannot reproduce this issue. (woks with settings as shown below)

Maybe the URLs at https://www.kwu-entsorgung.de/ have been changed?
WCS worked perfectly until last week!

No that's not it source not found: kwu_de indicates that the source_shell.py script did not find the custom_components/waste_collection_schedule/waste_collection_schedule/source/kwu_de.py file which is there since November 25th, 2022 so it must be some weird/broken installation. Can you check if this file exists?

image

@Jirka-Eh
Copy link
Author

Checked, the file exists!

image

@Jirka-Eh
Copy link
Author

I tested several german sources. Most of them I can't add!
Sources from other countries are working!

@meilon
Copy link
Contributor

meilon commented Oct 23, 2024

I have the same issue with awb_es_de
I configure it through yaml, the config hasn't been touched since I added them. I downloaded the WCS source locally and ran the test script after editing the test case in the awb_es_de.py to my settings and it returns the next pickups without issue.
The WCS files are where they are supposed to be, I don't know why (and sadly when after which update) it stopped working

@5ila5
Copy link
Collaborator

5ila5 commented Oct 23, 2024

Seems to be related to #2907

@Jirka-Eh
Copy link
Author

Ok, I have ICS Calendar installed.
But what can we do?

@ReneNulschDE
Copy link
Contributor

ReneNulschDE commented Oct 23, 2024

Confirm your ICS Calendar version. If it’s greater than 5.0.3, proceed. If not, share the version or any custom components you’re using.

For most users, the best option is to wait for a general solution (which requires an update of the underlying library, icalevents).

Note: I tested this only in my environment! Make sure to stop here unless you’ve made a backup and are fully prepared for potential issues.

You have to decide what is more important:

  1. The actual version of waste collection schedule
    or
  2. The actual version of ICS calendar

(Updated based on the comments below)
If you decide for option one:

  1. Use HACS to downgrade ICS calendar to 5.0.3 (5.0.4 introduced the minimum icalendar 6.0.0)
  2. Install, configure and start the addon "Advanced SSH & Web Terminal"
  3. Switch to the addon and run docker exec -it homeassistant /bin/sh to connect to a shell of the homeassistant docker container.
  4. Review the installed icalendar lib with pip list | grep ical, if icalendar is <=6.0.0 we need to uninstall it, if not - STOP and share the details here
  5. Uninstall the libs with pip uninstall icalendar icalevents (icalevents needs to be uninstalled to make sure that we get the right version on restart)
  6. Restart Home Assistant
  7. WCS should work again

if you decide for option two:
no change should be needed. WCS will not work.

@meilon
Copy link
Contributor

meilon commented Oct 23, 2024

I did it without having to give up either:

  1. Use HACS to downgrade ICS calendar to 5.0.3 (5.0.4 introduced the minimum icalendar 6.0.0)
  2. uninstalled icalendar and icalevents as suggested
  3. restarted Home Assistant

@Wilfried278
Copy link

If you decide for option one:

Use HACS to downgrade ICS calendar to 5.0.3 (5.0.4 introduced the minimum icalendar 6.0.0)
Install, configure and start the addon "Advanced SSH & Web Terminal"
Switch to the addon and run docker exec -it homeassistant /bin/sh to connect to a shell of the homeassistant docker container.
Review the installed icalendar lib with pip list | grep ical, if icalendar is >=6.0.0 we need to uninstall it, if not - STOP and share the details here
Uninstall the libs with pip uninstall icalendar icalevents (icalevents needs to be uninstalled to make sure that we get the right version on restart)
Restart Home Assistant

WCS don’t work !!!!!!!!!!

@ReneNulschDE
Copy link
Contributor

WCS don’t work !!!!!!!!!!

Hi @Wilfried278, what is not working and please explain your environment. Share the logs, share the output of the different steps. And please check your keyboard, looks like it is defect as it spawns exclamation marks...

@fsedarkalex
Copy link

fsedarkalex commented Oct 28, 2024

had the same issue with abfall_io_ics (as it uses the same lib). The downgrade of ics_calendar with removing the libraries worked.

@Lateadapter
Copy link

same issue here with ics ... (ICS calendar 5.0.3 works)

@nuki47
Copy link

nuki47 commented Nov 2, 2024

5. pip uninstall icalendar icalevents

This way works for a downgrade, BUT after some restarts version 5.0.4 comes back and WCS don't work

@Lateadapter
Copy link

  1. pip uninstall icalendar icalevents

This way works for a downgrade, BUT after some restarts version 5.0.4 comes back and WCS don't work

To be honest, I'm not experienced enough to follow the SSH approach. I solved it by going back with full backup shortly after introducing HA Core 2024.10.3. That brought back beside a couple of other "old" versions also a working WCS (at this time the root cause of ICS was not clear for me, but already in consideration). Therefore I did start step by step to reinstall all the newer integrations. After each step I performed a full HA-backup, till finally ICS 5.0.4. was the only remaining integration. Everything else was working.
I performed a backup, installed ICS 5.0.4. and WCS disappeared - so far so known...
But now the idea was to solve the problem with the intentionally prepared "full backup". But BIG SURPRISE: Restoring the backup did NOT solve the issue! Obviously a "full backup" is not as "full" as expected.
So I went back to the beginning and stopped before ICS 5.0.4. ... which now I will ignore ...

@Wilfried278
Copy link

For me, these repairs with older versions are not an option. I prefer to wait until a new, stable and functioning update comes out. The frequency of WCS updates within 48 hours shows that several problems still need to be resolved. So I'm still waiting.

@smmmo
Copy link

smmmo commented Nov 16, 2024

I run home assistant in docker using ghcr.io/home-assistant/home-assistant:2024.11 and have the same problem because of ICS calendar.

The workaround of downgrading ICS calendar did not work for me. Even after I completely uninstalled ICS calendar, icalendar 6.0.1 was reinstalled after restarting ha. I couldn't find any other usage of icalendar, only in WCS. I've used this to search:
grep -rnw './config/custom_components' -e 'icalendar'

However, the fix for me was to first uninstall icalendar as suggested above:
pip uninstall icalendar icalevents
and then force the installation of 5.0.13 (the last version before 6.x) by modifying custom_components/waste_collection_schedule/manifest.json:
"requirements": ["icalendar==5.0.13", "recurring_ical_events", "icalevents>=0.1.26,!=0.1.28", "beautifulsoup4", "lxml", "pycryptodome"],

I have no idea about Python, but maybe it's because WCS doesn't depend on any particular version of icalendar, so it always uses the latest version (if no other version is installed locally)?

@krajani786
Copy link

Hello,

I have deleted ICS Calendar integration, and i don't have any ical stuff at all. I deleted WSC and every local calendar i have. Advanced SSH & Web Terminal does not work for me, never loads, i get a 504 error. Maybe because i have HA in proxmox and not a docker. I run Google Calendar integration as well. I have about 8 ICS orphaned Calendars that are tied to the waste collection schedule integration (although its not installed at the moment) these cannot be deleted no matter what i try.
Everytime i try and add the WSC and choose my location i get unknown error. I do not know what logs you need or how to get them. please help.

@ReneNulschDE
Copy link
Contributor

ReneNulschDE commented Nov 27, 2024

Please describe in detail how you have installed your HA environment and share the System Information (Settings, System, Repair - click on three dots in the upper left corner and share the system information)

We need the output of the command pip list | grep ical of the environment where your HA system is running.

When the Adv SSH is not working, then please check the log of this addon. This is the most used addon for such tasks.

In case that you do not understand these tasks - I propose that you wait until a version of this component is available where the root cause (a problem in an sub-component) is solved.

@krajani786
Copy link

I have Proxmox installed, and installed HAOS in a VM. The VM is using a self install script i found online. I did get the ssh working and that info is also below. The system info is
`## System Information

version core-2024.11.3
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.12.4
os_name Linux
os_version 6.6.54-haos
arch x86_64
timezone America/Edmonton
config_dir /config
Home Assistant Community Store
GitHub API ok
GitHub Content ok
GitHub Web ok
HACS Data ok
GitHub API Calls Remaining 5000
Installed Version 2.0.1
Stage running
Available Repositories 1468
Downloaded Repositories 57
AccuWeather
can_reach_server ok
remaining_requests 10
Home Assistant Cloud
logged_in false
can_reach_cert_server ok
can_reach_cloud_auth ok
can_reach_cloud ok
Home Assistant Supervisor
host_os Home Assistant OS 13.2
update_channel stable
supervisor_version supervisor-2024.11.4
agent_version 1.6.0
docker_version 27.2.0
disk_total 30.8 GB
disk_used 25.4 GB
healthy true
supported true
host_connectivity true
supervisor_connectivity true
ntp_synchronized true
virtualization kvm
board ova
supervisor_api ok
version_api ok
installed_addons Terminal & SSH (9.15.0), Studio Code Server (5.17.3), File editor (5.8.0), ESPHome Device Compiler (2024.11.1), Google Assistant SDK (2.5.0), Home Assistant Google Drive Backup (0.112.1), Mosquitto broker (6.4.1), Piper (1.5.2), Get HACS (1.3.1), Network UPS Tools (0.13.2), Advanced SSH & Web Terminal (19.0.0)
Dashboards
dashboards 7
resources 44
views 16
mode storage
Recorder
oldest_recorder_run November 18, 2024 at 10:24 PM
current_recorder_run November 27, 2024 at 5:46 AM
estimated_db_size 1090.83 MiB
database_engine sqlite
database_version 3.45.3
Spotify
api_endpoint_reachable ok
`

And after i do the above ical stuff
➜ ~ pip list | grep ical ➜ ~ pip uninstall icalendar icalevents WARNING: Skipping icalendar as it is not installed. WARNING: Skipping icalevents as it is not installed. WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv ➜ ~

@ReneNulschDE
Copy link
Contributor

ReneNulschDE commented Nov 27, 2024

And do you really connected to the hass-container? Step 3?

Now restart your HA environment. And check again. In case it is not working, share the relevant entries out of the homeassistant.log

@krajani786
Copy link

Hey,

Thank you for your assistance. I did figure it out, simple enough i did not configure the add on properly. Although that did not fix the issue, i had no ical stuff installed. What i did, was the same thing someone mentioned above. I installed icalendar again, version 5.0.1 and did nothing with it. Then installed WSC again and it worked. This also got rid of all the ghost calendars.

@Kanecaine
Copy link

Kanecaine commented Dec 18, 2024

So yet we have a short-term workaround. Many thanks to everyone who contributed to finding the cause in the ics-calendar component!
The current solution is to downgrade ics-calendar to version 5.0.3. Ok, but there was already 5.0.4 and 5.1.0. It can't be the solution to leave this component on the old version forever. The maintainer of ics-calendar does not feel responsible for the fact that another component is disrupted by a dependency upgrade. For me as a user, this is quite understandable, because after all, the intention should always be to use up-to-date, bug-fixed and improved versions of a package.

Accordingly, I assume that the solution must be found in WCS so that WCS is compatible with current versions of icalendar. But I admit that I am not familiar with all the technical backgrounds. I do wonder why the dependencies on other packages can lead to such problems. There seems to me to be a fundamental problem in the architecture, or in the way it is used.

Hence my question: can WCS be made compatible with the current version of icalendar or can the latest compatible version be used / referenced by WCS (together with ics-calendar > 5.0.3)?

Alternatively, please coordinate with the maintainer of ics-calendar if the solution has to be found together.

Many thanks in advance!

@5ila5
Copy link
Collaborator

5ila5 commented Dec 21, 2024

The problem is, that we're using icalevents which makes it much easier to work with, especially for ICS files using reoccurring events. The best way to resolve this would be for the icalevents maintainer.
There already exists a pull request for this (jazzband/icalevents#145) which looks like a pretty easy fix, but it does not look like there is any movement at the moment.

We could integrate ship icalevents with the changes within this repo (As both are MIT license it would be possible) but not my favorite approach

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests