New Tagging Scheme + Issue Features (The-OpenROAD-Project#1216)

+ Tagging scheme is now YYYY.MM.DD, postfixed with rX for multiple releases on the same day to match internal Efabless utilities + Add git remotes to environment survey + Add feature to force reproducibles to be created for a specific script regardless of failure: see `docs/source/using_or_issue.md` for more info ~ Fix `generate_tag.py` - Remove debugging vestiges from `all.tcl`
MrShaTr · Jul 20, 2022 · 2f94230 · 2f94230
1 parent b52022f
commit 2f94230
Show file tree

Hide file tree

Showing 6 changed files with 72 additions and 30 deletions.
diff --git a/.github/scripts/generate_tag.py b/.github/scripts/generate_tag.py
@@ -25,7 +25,9 @@
 
 latest_tag = None
 latest_tag_commit = None
-for tag in gh.openlane.tags:
+commits_with_tags = gh.openlane.tags
+tags = [tag for _, tag in commits_with_tags]
+for tag in commits_with_tags:
     commit, name = tag
     latest_tag = name
     latest_tag_commit = commit
@@ -41,7 +43,12 @@
 else:
     now = datetime.datetime.now()
 
-    new_tag = now.strftime("%Y.%m.%d_%H.%M.%S")
+    time = now.strftime("%Y.%m.%d")
+    new_tag = time
+    release_counter = 0
+    while new_tag in tags:
+        release_counter += 1
+        new_tag = f"{time}r{release_counter}"
 
     print("Naming new tag %s." % new_tag)
 

diff --git a/.github/scripts/gh.py b/.github/scripts/gh.py
@@ -105,7 +105,7 @@ def git_command(*args):
         )[:-1]
 
     repo_url = git_command("remote", "get-url", "origin")
-    branch = git_command("branch", "rev-parse", "--abbrev-ref", "HEAD")
+    branch = git_command("rev-parse", "--abbrev-ref", "HEAD")
 
     os.environ["REPO_URL"] = repo_url
     os.environ["BRANCH_NAME"] = branch

diff --git a/docs/source/using_or_issue.md b/docs/source/using_or_issue.md
@@ -7,9 +7,26 @@ It creates a folder with all the needed files that you can inspect, then zip or
 When working with a proprietary PDK, also inspect the folder and ensure no proprietary data resulting ends up in there. This is *critical*, if something leaks, this scripts' authors take no responsibility and you are very much on your own. We will try our best to output warnings for your own good if something looks like a part of a proprietary PDK, but the absence of this message does not necessarily indicate that your folder is free of confidential material. 
 
 # Usage
-If you're using OpenLane 2021.12.17 or later (OpenLane 2022.06.21 or later for Magic,) chances are, `or_issue.py` was automatically run for you if Magic or OpenROAD fail. You'll find a message in the log that says something along the lines of: `Reproducible packaged: Please tarball and upload <PATH> if you're going to submit an issue.` The path will be under the current run_path, i.e., ./designs/<design>/runs/<run_tag>/issue_reproducible. You can then tarball/zip and upload that file.
+## Failures
+If you're using OpenLane 2021.12.17 or later (OpenLane 2022.06.21 or later for Magic,) chances, `or_issue.py` will **automatically** be run for you if Magic or OpenROAD exit with a non-zero code. 
+
+You'll find a message in the log that says something along the lines of: `Reproducible packaged: Please tarball and upload <PATH> if you're going to submit an issue.` The path will be under the current run_path, i.e., ./designs/<design>/runs/<run_tag>/issue_reproducible. You can then tarball/zip and upload that file.
+
+## Odd Behavior
+If the Tcl-based script doesn't fail outright, but simply exhibits weird behavior, starting OpenLane 2022.07.20, you can have the flow simply stop execution and package a reproducible instead.
+
+If you know the name of the script causing the issue, you can set the environment variable `CREATE_REPRODUCIBLE_FROM_SCRIPT` to the name of the script. For example, to quit on Magic's DRC script, you can set the variable as follows:
+
+```bash
+export CREATE_REPRODUCIBLE_FROM_SCRIPT=magic/drc.tcl
+./flow.tcl [...]
+```
+
+The flow will automatically quit right before executing any script matching `CREATE_REPRODUCIBLE_FROM_SCRIPT`. The last message printed will be something among the lines of `[INFO]: Reproducible packaged at '<PATH>'.`
+
+## Manually
+If neither option above works for you for some reason, there's always the hard way, and the hard way involves invoking the script manually.
 
-## Running or_issue.py manually
 You'll have to extract three key elements from the **verbose** logs (i.e. ./flow.tcl must be run with `-verbose`):
 * The Script Where The Failure Occurred -> script
 * The Final Layout Before The Failure Occurred -> input

diff --git a/env.py b/env.py
@@ -86,15 +86,15 @@ def issue_survey():
     final_report += textwrap.dedent(
         """\
         Kernel: %s v%s
-    """
+        """
         % (os_info.kernel, os_info.kernel_version)
     )
 
     if os_info.distro is not None:
         final_report += textwrap.dedent(
             """\
             Distribution: %s %s
-        """
+            """
             % (os_info.distro, (os_info.distro_version or ""))
         )
 
@@ -109,7 +109,7 @@ def issue_survey():
     final_report += textwrap.dedent(
         """\
         Python: v%s (%s)
-    """
+        """
         % (python_version, python_message)
     )
 
@@ -126,7 +126,7 @@ def issue_survey():
         final_report += textwrap.dedent(
             """\
             Container Engine: %s v%s (%s)
-        """
+            """
             % (os_info.container_info.engine, container_version, container_message)
         )
     elif os.path.exists(
@@ -162,7 +162,7 @@ def issue_survey():
         final_report += textwrap.dedent(
             """\
             OpenLane Git Version: %s
-        """
+            """
             % get_tag()
         )
 
@@ -189,9 +189,9 @@ def issue_survey():
             venv_ok = False
 
         alert = (
-            "pip:venv: " + "INSTALLED"
+            "python-venv: " + "INSTALLED"
             if venv_ok
-            else "NOT FOUND: needed for containerless installs"
+            else "NOT FOUND: Please install python-venv using your operating system's package manager."
         )
         final_report += "%s\n" % alert
         print(alert, file=alerts)
@@ -228,6 +228,12 @@ def issue_survey():
             ).decode("utf8")
 
             final_report += "---\nGit Log (Last 3 Commits)\n\n" + git_log
+
+            remotes = subprocess.check_output(["git", "remote", "-v", "show"]).decode(
+                "utf8"
+            )
+
+            final_report += "---\nGit Remotes\n\n" + remotes
         except subprocess.CalledProcessError:
             pass
 

diff --git a/scripts/tcl_commands/all.tcl b/scripts/tcl_commands/all.tcl
@@ -428,14 +428,10 @@ proc prep {args} {
     puts_info "Using configuration in '$config_file_rel'..."
     source_config -run_path $run_path $::env(DESIGN_CONFIG)
 
-    puts_info $::env(STD_CELL_LIBRARY)
-
     if { [info exists arg_values(-override_env)] } {
         load_overrides $arg_values(-override_env)
     }
 
-    puts_info $::env(STD_CELL_LIBRARY)
-
     # Diagnostics
     if { ! [info exists ::env(PDK_ROOT)] || $::env(PDK_ROOT) == "" } {
         puts_err "PDK_ROOT is not specified. Please make sure you have it set."

diff --git a/scripts/utils/utils.tcl b/scripts/utils/utils.tcl
@@ -186,8 +186,8 @@ proc run_tcl_script {args} {
 
     set_if_unset arg_values(-indexed_log) /dev/null
 
+    set create_reproducible 0
     set script [lindex $args 0]
-
     set tool $arg_values(-tool)
 
     if { $tool == "openroad" } {
@@ -211,20 +211,31 @@ proc run_tcl_script {args} {
 
     set script_relative [relpath . $script]
 
-    puts_verbose "Executing $tool with Tcl script '$script_relative'..."
+    set exit_code 0
 
-    set exit_code [catch {exec {*}$args} error_msg]
+    if { [info exists ::env(CREATE_REPRODUCIBLE_FROM_SCRIPT)] && [string match *$::env(CREATE_REPRODUCIBLE_FROM_SCRIPT) $script] } {
+        puts_info "Script $script matches $::env(CREATE_REPRODUCIBLE_FROM_SCRIPT), creating reproducible..."
+        set create_reproducible 1
+    } else {
+        puts_verbose "Executing $tool with Tcl script '$script_relative'..."
 
-    if { $exit_code } {
-        set print_error_msg "during executing $tool script $script"
+        set exit_code [catch {exec {*}$args} error_msg]
 
-        puts_err "$print_error_msg"
-        puts_err "Exit code: $exit_code"
-        puts_err "full log: [relpath $::env(PWD) $arg_values(-indexed_log)]"
-        puts_err "Last 10 lines:\n[exec tail -10 << $error_msg]\n"
+        if { $exit_code } {
+            set print_error_msg "during executing $tool script $script"
+            set log_relpath [relpath $::env(PWD) $arg_values(-indexed_log)]
+
+            puts_err "$print_error_msg"
+            puts_err "Log: $log_relpath"
+            puts_err "Last 10 lines:\n[exec tail -10 << $error_msg]\n"
+
+            set create_reproducible 1
+            puts_err "Creating issue reproducible..."
+        }
+    }
 
+    if { $create_reproducible } {
         save_state
-        puts_info "Creating reproducible..."
 
         set reproducible_dir $::env(RUN_DIR)/issue_reproducible
         set reproducible_dir_relative [relpath $::env(PWD) $reproducible_dir]
@@ -242,13 +253,18 @@ proc run_tcl_script {args} {
             lappend or_issue_arg_list $::env(CURRENT_DEF)
         }
 
-        if {[catch {exec -ignorestderr python3 $::env(SCRIPTS_DIR)/or_issue.py {*}$or_issue_arg_list} result] == 0} {
-            puts_info "Reproducible packaged: Please tarball and upload $reproducible_dir_relative if you're going to submit an issue."
-        } else {
+        if {![catch {exec -ignorestderr python3 $::env(SCRIPTS_DIR)/or_issue.py {*}$or_issue_arg_list} result] == 0} {
             puts_err "Failed to package reproducible."
+            flow_fail
         }
 
-        flow_fail
+        if { $exit_code } {
+            puts_info "Reproducible packaged: Please tarball and upload '$reproducible_dir_relative' if you're going to submit an issue."
+            flow_fail
+        } else {
+            puts_info "Reproducible packaged at '$reproducible_dir_relative'."
+            exit 0
+        }
     }
 }