Krzysztof Kaczor

Passionate Software Developer

Get child elements using Windows Automation

on 24. July 2015

During work on STAMP for Windows (it allows you to move your Spotify music library to Apple Music) I needed a reliable way of using other application UI in automated way. Basically I needed to find UI elements like buttons, grids and perform some actions on them.

On Mac this is easy peasy - you have Apple Script which was crafted exactly to do stuff like that. But on Windows there is no easy way to accomplish this goal. You can use very low level API like P/Invoke or try Windows Automation or use third party tools like AutoIt or AutoHotKey to do that. Unfortunately in long term each of this solution have failed me. For example neither AutoIt neither AHK could recognize components of iTunes window. To be honest iTunes UI behaves weird compared to other windows apps and I am sure that simpler solutions would work when dealing with other application.

In the end I used mix of P/Invoke magic (mostly to send communicates to other application like sending key strokes or clicks) with layer of Windows Automation to provide reliable solution. Here’s how to deal with Windows Automation weirdness.

Windows Automation API consists of FindAll method to get child/descendants of a given element. It is essential in walking complicated UI elements tree. For example:

rootElement.FindAll(TreeScope.Children, PropertyCondition.TrueCondition);

will return collection of all children elements of given rootElement. You can of course do a lot more with that method like selecting only nodes that satisfy given conditions etc. But the problem that I experienced was that it sometimes simply fails. After researching this problem I have discovered that this method caches found UI Elements and that was root of my problems.

After that discovery I decided to use RawTreeWalker which gives programmer more power on what’s going under the hood. Putting it as extension method of AutomationElement results in this simple solution:

public static List<AutomationElement> GetAllChildren(this AutomationElement element)
{
        var allChildren = new List&lt;AutomationElement&gt;();
        AutomationElement sibling = TreeWalker.RawViewWalker.GetFirstChild(element);

        while (sibling != null)
        {
            allChildren.Add(sibling);
            sibling = TreeWalker.RawViewWalker.GetNextSibling(sibling);
        }

        return allChildren;
}

which you can use like this:

rootElement.GetAllChildren();

Similarly you can craft recursive method to get all descendants:

public static List&lt;AutomationElement&gt; GetAllDescendants(this AutomationElement element, int depth = 0, int maxDepth = 3)
{
        var allChildren = new List<AutomationElement>();

        if (depth > maxDepth){
            return allChildren;
        }

        AutomationElement sibling = TreeWalker.RawViewWalker.GetFirstChild(element);

        while (sibling != null)
        {
            allChildren.Add(sibling);
            allChildren.AddRange(sibling.GetAllDescendants(depth+1, maxDepth));
            sibling = TreeWalker.RawViewWalker.GetNextSibling(sibling);
        }

        return allChildren;
}

After getting these elements you can filter, map or do whatever you want with them using LINQ (instead of built in Windows Automation conditions):

//select child elements of rootElement that has at least 5 children
element.GetAllChildren().Where(e => e.GetAllChildren().Count >= 5);

Maybe using LINQ is not as robust and performant as using Windows Automation selecting capabilities but it is a way more reliable. We use this approach in STAMP and it works much better then any previous solution that we were testing.