Robotic Process Automation (RPA) is a technology which automates mouse and keyboard interactions by means of a software (SW) robot to remove intensive routines. The current generation of RPA tools is driven by predefined rules and manual configurations made by expert users rather than automated techniques.

SmartRPA is a cross-platform tool that tackles such issues. It allows to easily record event logs and to automatically generating executable RPA scripts that will drive a SW robots in emulating an observed user behavior (previously recorded in dedicated UI logs) during the enactment of a routine of interest.

SmartRPA has been developed at DIAG, Department of Computer, Control, and Management Engineering Antonio Ruberti in Sapienza University of Rome by:

• Simone Agostinelli
• Marco Lupia
• Andrea Marrella

Many thanks also to Tom Hohenadl (Technische Hochschule Ingolstadt) and Antonio Martínez-Rojas (Universidad de Sevilla) for the ongoing updates of the tool.

A Screencast of the tool is available on [Vimeo]. Additionally, a dedicated video showcasing both the screenshot and supervision features is included here [Vimeo].

The architecture of SmartRPA integrates five main SW components.

Key features include:

• Action Logger, log user behaviour, take screenshots, tag actions, supports wide range of applications, cross-platform;
• Log Processing, generates both CSV and XES event log;
• Event abstraction, abstracts events to a higher level;
• Process Discovery, selects the most suitable routine variant to automate and generates high-level flowchart diagram, thus skipping completely the manual modeling activity;
• Decision Points, discover differencies between multiple traces in a process and build a new routine based on user decisions;
• RPA, implements and enacts a SW robot emulating a routine reflecting the observed behavior (either the most frequent one or the one based on decision points). Available both as a cross-platform Python script and as a UiPath project.

A list of events supported by the Action Logger is available in SmartRPA_events.pdf.

The full documentation of the tool is available here. A user manual for SmartRPA is available in this repository here.

The following dependencies are required to enable all functionalities within SmartRPA. Check all boxes and you are able to log user actions, discover process models, and emulate RPA bots in UiPath and Automagica automatically:

• Prerequisites
  • Visual Studio C/C++ Build Tools (Windows Only)
  • Graphviz Windows/Mac
  • Automagica
  • Chromedriver for Edge/Chrome
  • UiPath Studio
• Action Logger dependencies
  • Python Requirements
  • Browser extension
• Excel AddIn (MacOS Only)
• LLM Tagger

Furthermore, there are possible installation issues that can arise with the installation of the prerequisites or libraries. You can find fixes to the most common issues in the section Installation Errors .

• On Windows, Visual Studio C/C++ Build Tools must be installed. It is vital to install all C++ related development tools listed here:

  • Windows 10/11 SDK
  • Visual C++ tools for CMake
  • C++ x64/x86 build tools

Install the latest version of graphviz. Make sure to add it to system PATH. Detailed instructions here.

• On MacOS:

  1. Make sure you installed brew package manager
  2. Check [brew] installation for graphviz with brew install graphviz
  3. Check pm4py installation and run pip3 install pm4py==1.5.0.1 if necessary

Python ≥ 3.7 (64bit) is required. Python 3.12 is recommended.

pip3 install -r src/requirements.txt

We have specified three requirements files for distinct Python versions. We recommend using the specified requirements file. The requirements.txt is for Python 3.12.

pip3 install -r src/requirements308.txt

pip3 install -r src/requirements310.txt

To run the generated RPA scripts you must install automagica module available in the libraries directory.

pip3 install src/libraries/smartRPA-automagica-2-1-12.zip

pip3 install src/libraries/Automagica-2.0.25-py3-none-any.whl`

Required for executing the emulated Automagica Python bots in Chrome and Edge. To select the correct version of the pip installation for the Chromedriver go to the Chrome or Edge settings/help. Get the version number, e.g., Version 124.0.2478.97. Navigate to PyPi Chromedriver History. Select the release with a same main number, e.g., chromedriver-binary 124.0.6367.207.0, and install this version using pip, i.e.,

pip install chromedriver-binary==124.0.6367.207.0

The generated UiPath project requires UiPath Studio, available at https://www.uipath.com/product/studio.
Without UiPath Studio you cannot import and execute the generated UiPath bots from SmartRPA.

The browser extension is required to log browser events. It is available in src/extensions/browserlogger and supports 4 major browsers:

• Google Chrome: load unpacked browserlogger directory in chrome://extensions/

• Mozilla Firefox: install browserlogger.xpi in about:addons

• Microsoft Edge (chromium): load unpacked browserlogger directory in edge://extensions/

• Opera: load unpacked browserlogger directory in opera:extensions

Currently the browser extension requires the developer mode to be active in your browser. Please check the browser documentation on how to enable the developer mode.

Enabling the Browser Logger

Once the main logger is running, you must click on the browser extension to enable it.

Inactive Browser Logging	Active Browser Logging

The excel addin is required to log Excel events only on MacOS.

Node.js must be installed to run this addin.

cd src/extensions/excelAddinMac
npm install # install dependencies
npm start   # sideload Add-in
npm stop    # stop server

python3 main.py

A detailed screencast of the tool is available on [Vimeo] including the core functions of recording, analyzing, and emulating RPA bots. The resulting event log will be saved in logs/ directory. If the screenshot feature is active the screenshots are stored in the Screenshots directory.
SmartRPA generates two types of SW Robots in the RPA/ directory:

1. a cross-platform executable Python script, available on both Windows and MacOS
2. a UiPath project, available only on Windows

The advantages of the UiPath integration is that the generated SW Robot can be easily customized by the end user.

NOTE: In the Action Logger, when selecting a Microsoft Office program to log, it will automatically be opened. This is required to correctly handle events. The opened window should not be closed until logging is completed.

LLMs allow are capable of Business Process Management tasks and therefore can be used to semantically tag data. We integrated such a feature in the processing menu.

• Open the Processing/LLM Tagging Menu
• Store your Gemini API code. A manual to obtain an Gemini API key is found here.
• Select the UI log you want to semantically tag
• Start the promoting to generate short text descriptions of the activities conducted

This feature is currently in Beta as the prompt results sometimes to not contain proper JSON responses for parsing or long logs cannot be processed accuratly.

If you only want the action logger to record event logs without further analysis, you can find it here . Please note that this logger is associated with version 1.0 of SmartRPA and has not been updated with the new functionalities (Supervision Feature, Screenshot Feature, Updated Python Libraries) of SmartRPA 2.0.

Could not save image: Command '[WindowsPath('dot'), '-Kdot', '-Tpdf', '-O', 'tmp24z1pppy.gv']' returned non-zero exit status 1. [stderr: b'There is no layout engine support for "dot"\r\nPerhaps "dot -c" needs to be run (with installer\'s privileges) to register the plugins?\r\n'] 

conda install conda-forge::python-graphviz conda-forge::graphviz=2.46.1

py main.py

sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

export CPATH=`xcrun --show-sdk-path`/usr/include

pip3 install --pre --extra-index https://pypi.anaconda.org/scipy-wheels-nightly/simple scikit-learn

set LIB=C:\Program Files\OpenSSL-Win64\lib;%LIB%
set INCLUDE=C:\Program Files\OpenSSL-Win64\include;%INCLUDE%
pip3 install libraries/Automagica-2.0.25-py3-none-any.whl

• Agostinelli S., Lupia M., Marrella A., Mecella M.: Reactive Synthesis of Software Robots in RPA from User Interface Logs. Accepted at Computers in Industry (Elsevier), 2022
• Agostinelli, S., Lupia, M., Marrella, A., Mecella, M.: SmartRPA: A Tool to Reactively Synthesize Software Robots from User Interface Logs. In: 33rd Int. Conf. on Advanced Information Systems Engineering (CAiSE Forum). pp. 137-145 (2021)
  • The synthetic UI logs generated for the test are available at: https://tinyurl.com/yyk68psx.
  • The complete results can be analyzed at: https://tinyurl.com/y55v56qa.
• Agostinelli, S., Lupia, M., Marrella, A., Mecella, M.: Automated Generation of Executable RPA Scripts from User Interface Logs. In: 18th Int. Conf. on Business Process Management (RPA Forum). pp. 116-131 (2020)

This work has been partially supported by the PNRR MUR project PE0000013-FAIR