Abusing robotframework-datadriver for fun and profit

After progressing from duplicating test steps to using test templates, you went a step further and starting using CSV files and datadriver. But did you know that datadriver includes a reader_class initialization parameter that you can (ab)use to generate tests from directory listings and REST APIs?

This is a 5-minute lightning talk to get people thinking about how they can use datadriver to outsource the source of truth about some of their test suites to either a directory of files or event a REST API such as from TestRails or JIRA.

After progressing from duplicating test steps to using test templates, you went a step further and started using CSV files and robotframework-datadriver. But did you know that datadriver includes a reader_class intitalization parameter that you can (ab)use to generate tests from directory listings and REST APIs?

I didn’t either, until I found myself writing a lot of trivial CSV files that looked like this:

*** Test Cases ***,${file_name}
my test name,my_test_name.json
another test name,another_test_name.json
yet another test name,yet_another_test_name.json

And then finding that this CSV file would get out of sync with the actual test JSON files. What if “my test name” was removed because it was no longer valid? What if “the best test yet” was added, but not included in the CSV file?

There were a few approaches we could take:

• Generate the CSV file during some earlier part of the CI/build process
• Generate the CSV file with a –variable-file script…
• Generate the CSV file with datadriver’s config_keyword initialization property

But this still wasn’t fully satisfying, because really, this CSV file did not need to exist. So, I dug into datadriver a bit further and starting using the reader_class initialization property and a trivial helper class:

*** Resources ***
Library    DataDriver    file=/path/to/my/files/**.json
...    reader_class=DataDriver.GlobDataReader 
...    file_search_strategy=None 

And all you really have to do is implement a get_data_from_source method and you are off and running. For our “trivial” example:

var_name = self.kwargs["var_name"] if "var_name" in self.kwargs else "${file_name}"
self._analyse_header(["*** Test Cases ***", var_name])
for f in glob.glob(self.file):
    self._read_data_from_table([os.path.basename(os.path.splitext(f)[0]),os.path.abspath(f)])
return self.data_table

Et voilà! Now:

My Test Template
    [Arguments]       ${file_name}
    My Test Keyword   ${file_name}

Your directory of JSON tests are heading to your test keyword for final disposition.

But wait! you say. What about tags? Documentation? Well, you’re already in Python code. You could embed these in the JSON files (or YAML files, or…) but why not just use requests to grab that metadata via REST from TestRails or JIRA?

GET index.php?/api/v2/get_suites/1
GET index.php?/api/v2/get_suite/2
GET index.php?/api/v2/get_cases/1&suite_id=2&milestone_id=3&priority_id=4...

This also has the advantage of getting rid of painstaking dual tag maintenance and making your PHB happy by making TestRails the source of truth about which tests to run where and when.

You could even grab data file(s) themselves:

GET index.php?/api/v2/get_attachments_for_case/:case_id
GET index.php?/api/v2/get_attachment/:attachment_id

Making TestRails the source of truth about the test case data file(s) themselves. Personally, I’d rather have these in Git and on my local filesystem, but hey: Whether it’s a good idea or not, Robot Framework will let you do it.

Thank you for your time.

I’ve submitted a pull request to the robotframework-datadriver project with my GlobDataReader class:

https://github.com/Snooz82/robotframework-datadriver/pull/34

And plan to submit a pull request for a REST API approach along with a TestRails API consumer.

There aren’t really any technical requirements for this, as it is intended as a 5-minute lightning talk to get people thinking about how they can use datadriver to outsource the source of truth about some of their test suites to either a directory of files or event a REST API such as from TestRails or JIRA.