R & Python Integrations Release

TagUI is a general purpose tool for automating web interactions

• allow snapshot of desktop or browser UI
• vision step for custom commands to Sikuli
• Tesseract OCR for steps read / show / save
• integration with R & Python for ML / big data
  • for details on R integration, see readme here
  • for details on Python integration, see here

TagUI is now maintained by AI Singapore (aisingapore.org), a national initiative to build local artificial intelligence capabilities. The intention is to add AI capabilities to TagUI while keeping it open-source.

Native Languages Release

TagUI is a general purpose tool for automating web interactions

• #68 write and run automation flows in different human languages (more info)
• #66 add run step for running macOS/Linux OS-level commands (more info)
• #35 address Chrome integration bug when no reply received from Chrome
• speed option to speed up batch automation (and skip restarting of Chrome)
• print step to display output on next line for consistent TagUI steps behavior

Test Automation Release

TagUI is a general purpose tool for automating web interactions

• #65 enabled file downloads for headless Chrome
• #64 fixed popup step for Chrome protocol changes
• #63 extending check step for auto test assertions
  • added url(), title(), text(), timer(), count('element'), visible('element') functions
  • in addition to existing present('element') function, for easy access to properties
  • these webpage properties can be used in conditions or steps such as check or echo

This release makes it possible for headless and visible Chrome test automation, with integration to CI/CD tools such as Jenkins for continuous integration. CasperJS does not support Chrome, thus the check step is now extended to perform test assertions automatically that work for Chrome. Eg:

// check that the page has the text My Portfolio
check text() contains 'My Portfolio' | 'page text contains My Portfolio' | 'page text does not contain My Portfolio'

// check that number of header menu items are more than or equals to 6
check count('uh-tb-') more than or equals to 6 | 'header menu items >= 6' | 'header menu items < 6'

Leveraging on TagUI's built-in integration with Chrome and CasperJS testing framework, a XUnit XML file can now be generated even for tests done on headless or visible Chrome browser. More details can be found at readme testing section. The flow sample 7_testing shows how above new functions and natural language conditions can be used in check step, or normal TagUI steps.

Pre-AI Release

TagUI is a general purpose tool for automating web interactions

• load step to load file content into variable
• pass dom_json variable(s) for use in dom step
• select step supports [clear] keyword to clear selection
• fix table step for tables having rows with header and data cells
• half Sikuli polling to 500 ms to speed up visual automation
• support edge case of having if block within for loop block
• Chrome extension records text entry after tab to next field
• add link to video demo for command line assistant
• general update of readme to improve clarity

Release all additions and changes since August, before adding AI integrations. Yandex CatBoost and Google deeplearn.js identified as potential integrations to support machine learning capabilities.

CatBoost uses gradient boosting to solve categorization problems in a computationally efficient way. deeplearn.js uses Google Chrome browser's WebGL engine to leverage graphic card hardware for deep learning computations. Initial assessment does not show technical roadblocks yet, the bigger challenge for now is to integrate them in meaningful ways.

I/O Improvements Release

TagUI is a general purpose tool for automating web interactions

• snap webpage to pdf file (PhantomJS / Headless Chrome)
• write step to append new line to file (dump step overwrites)
• table step to save simple html table to csv (non-CSS selector)
• auto-transpose datatable (save csv as flowname_transpose.csv)
• support datatable cells with commas (remove false-positive check)
• present() function to check element (can combine with if condition)
• Chrome extension note down element menu (easier to note element)

Been holding back implementing some of these changes in order to avoid attracting users. I’m happy with current user growth, growing faster will spread time thinly across much more edge cases needs.

Google just launched Puppeteer (check it out, new tool but great API!), I think most developers would be busy now exploring it than other web automation tool, so I feel safe to add these features now.

CLI Assistant Release

TagUI is a general purpose tool for automating web interactions

• natural-language-like syntax command line assistant
• intuitive conversational way to invoke automation flows
• this is not AI or smart, just an alternative way to call flows
• baseline option to capture run benchmark to baseline folder
• upload step to upload file to web element (css selector only)
• timeout step to change how long before step errors out
• enabled navigation of url containing dynamic variables
• for type / enter step, added [clear] keyword to clear field
• enable frame / popup step in for loop (single next step)
• improved handling and error-checking of { and } steps
• setting variable with or without putting ; at the end of line
• improved live mode, and live url navigation for chrome

General Improvements / Fixes

TagUI is a general purpose tool for automating web interactions

• add recommended resources section in readme developers reference
• update sample flows 6_datatables, add misc examples with 9_misc
• sanity check for prevention of shell code injection by web service
• add license file for php websocket just in case online url removed
• fix for loop for number variable to be correctly accessed in loop
• fix automation exit status display issue with some types of loops

Chrome Integration Release

TagUI is a general purpose tool for automating web interactions

• TagUI steps can now automate Chrome browser
• directly automate visible or headless Chrome
• websocket connection on separate PHP thread
• communication via Chrome Debugging Protocol
• chrome_step(method, params) to send and receive

List of custom CasperJS methods to communicate with Chrome

• back, capture, captureSelector, click, debugHTML, download, evaluate, exists, fetchText, forward, getCurrentURL/HTML/Title, mouse.xxx, reload, sendKeys, waitForPopup, withFrame, withPopup

Visual Automation Release

TagUI is a general purpose tool for automating web interactions

• built-in integration with Sikuli for visual automation
• identifying web and desktop elements using images
• {} step to set code block for if/for/while/popup/frame
• advance api step for REST methods / headers / body
• Chrome extension element identification improvements
• upload option to upload flow result to hastebin.com
• support [enter] keyword for type / enter step (enter key)

Live Mode Release

TA.Gui is a tool for non-developers and business users to automate web apps

• live mode to try TA.Gui steps or Javascript code in real-time
• make tagui runnable from any folder in command prompt
• support running online automation flow hosted at an URL
• popup step to run next step or JS code on a popup window
• dom step to run JS code in Document Object Model context
• retrieving page source (raw html) for save/show/read steps
• parsing code optimization and streamline documentation