Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Script Stuck When Running build_map.py with mosstool #4

Open
HlllMan opened this issue Dec 14, 2024 · 6 comments
Open

Script Stuck When Running build_map.py with mosstool #4

HlllMan opened this issue Dec 14, 2024 · 6 comments
Labels
more-information-needed label for auto no-response

Comments

@HlllMan
Copy link

HlllMan commented Dec 14, 2024

Describe the bug
The process for building the map using mosstool does not complete successfully. It appears to be stuck or repeatedly outputs warnings without generating the expected results.

To Reproduce
Steps to reproduce the behavior:
1. Clone the repository and set up the required environment using conda and dependencies as instructed.
2. Prepare the input files: sydney_topo.geojson and sydney_aois.geojson (both are in map_cache).
3. Run the script build_map.py with the provided code snippet:

from mosstool.map.osm import RoadNet, Building
rn = RoadNet(
proj_str="+proj=merc +lat_0=-33.882244 +lon_0=151.111935",
max_latitude=-33.721288,
min_latitude=-34.0432,
max_longitude=151.243988,
min_longitude=150.839882
)
roadnet = rn.create_road_net("map_cache/sydney_topo.geojson")

building = Building(
proj_str="+proj=merc +lat_0=-33.882244 +lon_0=151.111935",
max_latitude=-33.721288,
min_latitude=-34.0432,
max_longitude=151.243988,
min_longitude=150.839882
)
aois = building.create_building("map_cache/sydney_aois.geojson")

from mosstool.map.builder import Builder
builder = Builder(
net=roadnet,
aois=aois,
proj_str="+proj=merc +lat_0=-33.882244 +lon_0=151.111935"
)
m = builder.build("example")

4.	The script outputs warnings such as “line outer ways make more than one polygon” and “exist both ring and line outer ways,” and it appears stuck indefinitely.

Expected behavior
The script should process the input files and produce the expected results, such as generating the built map or any relevant output.

Screenshots
Attached are the relevant screenshots:
1. Repeated warnings in the terminal.
image

2.	The script used for testing.

image

Versions
• OS: Linux Ubuntu 22.04
• Python Version: Python 3.9 (installed via Conda)
• CUDA Version: CUDA 12.2
• Tool Version: mosstool 1.2.0

Additional context
• Input files used: sydney_topo.geojson and sydney_aois.geojson.
• Temporary issues like insufficient disk space were resolved during the testing. The remaining problem persists even after ensuring proper configurations for cache and environment.
• Environment was activated using Conda with sufficient privileges.

@Sweetnow
Copy link
Member

@chenchenplus

@chenchenplus
Copy link

Thank you for raising this issue!

It seems that the problem might be due to the logging level not being set in the code, which results in only warnings being displayed while info-level updates that show the progress of the code are hidden. To resolve this, you can try adding the following lines to your code:

import logging
logging.basicConfig(level=logging.INFO)

This should allow you to see more detailed information about the code's execution and help determine if it is getting stuck at any point. Please let us know if this helps or if there's anything else we can assist you with.

@HlllMan
Copy link
Author

HlllMan commented Dec 16, 2024

Updated Feedback

Thank you for your suggestion! I have added import logging and logging.basicConfig(level=logging.INFO) as you recommended. The updated code is as follows:

import logging
logging.basicConfig(level=logging.INFO)

from mosstool.map.osm import RoadNet, Building

Set the latitude/longitude range and projection string for Sydney

rn = RoadNet(
proj_str="+proj=tmerc +lat_0=-33.882244 +lon_0=151.111935", # Projection string centered at Sydney
max_latitude=-33.721288, # Northern boundary of the Sydney area
min_latitude=-34.0432, # Southern boundary of the Sydney area
max_longitude=151.243988, # Eastern boundary of the Sydney area
min_longitude=150.839882 # Western boundary of the Sydney area
)

Create road network data

logging.info("Starting road network creation...")
roadnet = rn.create_road_net("map_cache/sydney_topo.geojson")
logging.info("Road network creation completed.")

Create building data

logging.info("Starting building data creation...")
building = Building(
proj_str="+proj=tmerc +lat_0=-33.882244 +lon_0=151.111935", # Use the same projection string
max_latitude=-33.721288,
min_latitude=-34.0432,
max_longitude=151.243988,
min_longitude=150.839882
)
aois = building.create_building("map_cache/sydney_aois.geojson")
logging.info("Building data creation completed.")

from mosstool.map.builder import Builder

Build the map

logging.info("Starting map construction...")
builder = Builder(
net=roadnet,
aois=aois,
proj_str="+proj=tmerc +lat_0=-33.882244 +lon_0=151.111935", # Projection string for Sydney
)
m = builder.build("example")
logging.info("Map construction completed!")

Results

•	On Mac: The code throws an error NameError: name 'aois_to_merge' is not defined during execution. See Figure 1 for the specific traceback.

image
• On Linux: The code gets stuck at the following step with no further output:
image

INFO:root:Merging Point AOI
INFO:root:Extending aois_poi to Square
INFO:root:Merging Covered Aoi

Refer to Figure 2 for the corresponding logs.

Issue Description

1.	Mac: The error aois_to_merge is not defined suggests a potential issue within the mosstool package where the variable may be missing or improperly referenced.
2.	Linux: The code stalls indefinitely at the “Merging Covered Aoi” step without proceeding further or throwing an error.

Expected Behavior

The code should successfully complete the map construction and log:

INFO:root:Map construction completed!

Environment Details

•	OS: MacOS 14.1.2, Ubuntu 22.04
•	Python Version: Python 3.9
•	mosstool Version: Latest
•	CUDA Version: CUDA 12.0 (Linux Server Only)

Additional Notes

Could you please help clarify:
1. Why aois_to_merge is not defined in the mosstool library on Mac?
2. Why the process hangs at “Merging Covered Aoi” on Linux?

Let me know if further debugging or additional logs are required. Thank you!

@chenchenplus
Copy link

Hi,

  1. For the aois_to_merge issue on Mac: We removed global variables in version 1.2.0 for inter-process communication, which previously caused similar errors. Please ensure you're using version 1.2.0 or later on Mac.
  2. Regarding the hang at "Merging Covered Aoi" on Linux: This step is CPU-intensive and cost a lot of time, starting from hours. It merges adjacent AOIs in a larger one and doesn't affect the simulation. You can skip this by setting merge_aoi = False during builder initialization to avoid delays.
logging.info("Starting map construction...")
builder = Builder(
    net=roadnet,
    aois=aois,
    merge_aoi=False,
    proj_str="+proj=tmerc +lat_0=-33.882244 +lon_0=151.111935", # Projection string for Sydney
)
m = builder.build("example")
logging.info("Map construction completed!")

@HlllMan
Copy link
Author

HlllMan commented Dec 16, 2024

Bug Report Summary

Issue 1: When running the Builder class without merge_aoi=False, a TypeError occurs on MacOS.
Issue 2: Even with merge_aoi=False and workers=0 (to disable multiprocessing), a similar issue arises, indicating an error in _add_poly_aoi_unit() or _find_aoi_parent_unit() during the map building step.

Issue Description

•	Platform: MacOS
•	Version: mosstool 1.2.0
•	Python Version: Python 3.9
•	Error Message:
•	Before adding merge_aoi=False and workers=0:

TypeError: _add_poly_aoi_unit() missing 1 required positional argument: 'arg'

•	After adding merge_aoi=False and workers=0:

TypeError: _find_aoi_parent_unit() missing 1 required positional argument: 'i_aoi'

Steps to Reproduce

1.	Install mosstool version 1.2.0 on MacOS.
2.	Use the following Python code with merge_aoi=False and workers=0:

import logging
from mosstool.map.osm import RoadNet, Building
from mosstool.map.builder import Builder

设置日志级别

logging.basicConfig(level=logging.INFO)

初始化路网和建筑数据

rn = RoadNet(
proj_str="+proj=tmerc +lat_0=-33.882244 +lon_0=151.111935",
max_latitude=-33.721288,
min_latitude=-34.0432,
max_longitude=151.243988,
min_longitude=150.839882
)

路网数据

logging.info("开始创建路网数据...")
roadnet = rn.create_road_net("map_cache/sydney_topo.geojson")
logging.info("路网数据创建完成.")

建筑数据

logging.info("开始创建建筑数据...")
building = Building(
proj_str="+proj=tmerc +lat_0=-33.882244 +lon_0=151.111935",
max_latitude=-33.721288,
min_latitude=-34.0432,
max_longitude=151.243988,
min_longitude=150.839882
)
aois = building.create_building("map_cache/sydney_aois.geojson")
logging.info("建筑数据创建完成.")

构建地图

logging.info("开始构建地图...")
builder = Builder(
net=roadnet,
aois=aois,
merge_aoi=False, # 跳过 AOI 合并步骤
workers=0, # 禁用多进程
proj_str="+proj=tmerc +lat_0=-33.882244 +lon_0=151.111935"
)

try:
m = builder.build("example")
logging.info("地图构建完成!")
except Exception as e:
logging.error(f"地图构建过程中出现错误: {e}")

3.	Run the code, observe the error.

Expected Behavior

The Builder.build() function should construct the map successfully without requiring the arg parameter for _add_poly_aoi_unit() or _find_aoi_parent_unit().

Actual Behavior

The following errors occur:
1. TypeError: _add_poly_aoi_unit() missing 1 required positional argument: 'arg'
2. TypeError: _find_aoi_parent_unit() missing 1 required positional argument: 'i_aoi' (after setting merge_aoi=False and workers=0).

Screenshots

1.	Error Before Adding merge_aoi and workers:

image

2.	Error After Adding merge_aoi and workers:

image

Additional Context

•	I am running the code on MacOS.
•	I have verified that the mosstool version is 1.2.0.
•	The issue persists even after setting merge_aoi=False and disabling multiprocessing with workers=0.

@chenchenplus
Copy link

Hello, I think the mosstool version used in your notebook is not 1.2.0. The codes shown in your screenshot
1.Error Before Adding merge_aoi and workers
is not the latest version (https://github.com/tsinghua-fib-lab/mosstool/blob/v1.2.0/mosstool/map/_map_util/aois/append_aois_matcher.py#L1542).
The version of mosstool used within the Jupyter Notebook might not align with the one available in your terminal. This difference occurs based on the kernel you select inside the Jupyter Notebook.

@Sweetnow Sweetnow added the more-information-needed label for auto no-response label Dec 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
more-information-needed label for auto no-response
Projects
None yet
Development

No branches or pull requests

3 participants