Updated Wiki: Documentation

Introduction
The goals of the project are:

• understanding reports relationships and consistency rules
• creating thorough test scenarios as standalone files which could be used as a part of continuous integration process
• simulating user behavior (mocking), covering functional and UI testing

How it works
Each reports server has SOAP end points available both modes Native and SharePoint modes. These endpoints are requested by the application based on settings provided in Settings.xml file. User (executing account) should have access permission to these services. The application requests web services which generate XML view of the report. The XML data is used in test cases provided by application settings. The file could be found on "Source Code" tab under the root project and named Settings.xml
If settings are not correct or communication with server is brocken, the application will write error in Error.log text file in the same folder and then generate excepton.
Here is application settings hierarchy:

• Settings
  • ReportServer
    • Folder
      • Report
        • Params
          • Param
        • TestCases
          • TestCase
            • DrillDownReport
              • Params
                • Param
        • LinkedReports
          • LinkedReport
            • Params
              • Param

Here is xsd schema

The functionality of test cases is defined by number of testing techniques implemented by application. In other words, testing algorithm should be able to estimate specific condition and return true or false. For example:

• Check if this field is blank
• Compare total from summary report with detail report total
• Generate html report for test suite
• Render report with specific parameters and serialize as a file
• Create linked report scenarios

The user can combine algorithms in settings file as testing scenarios.
The number of testing algorithms is a key feature of the project. It is a subject to improve solution.
Another typical task is deployment of linked reports with specific parameters. Some of the parameters should be hidden, some not but should have specific default values. The other parameters should be accurately inherited from master report. For that reason <LinkedReports> section was added as a part of <Report> element.
It allows to create test scenatio with number of <LinkedReport> items.

• <LinkedReports>
  • <LinkedReport Path="/AdventureWorks Sample Reports/Linked Reports/Bike Sales 2007" Description="Shows Product Line Report data with predefined parameters">
    • <Params>
      • <Param Name="StartDate" DefaultValues="2007/01/31" Hide="True"/>
      • <Param Name="EndDate" DefaultValues="2007/03/01" Hide="True"/>
      • <Param Name="ProductCategory" DefaultValues="1" />
      • <Param Name="ProductSubcategory" DefaultValues="1,2,3" />
    • </Params>
  • </LinkedReport>
• </LinkedReports>

Finally, new report is created in specific folder with new parameters (the highlighted check boxes with blank values are hidden from end user)



How to set up
Each element has attributes (Mode, Path, Name, etc.) which would be setup before running the tests.
ReportServer element attributes can have the following values:

• Path="http://<ServerName>/<ReportServer_Instance>/"
• Mode="Native|SharePoiunt"
• ParameterLanguage="en-us|en-au|..."
• UserName="@UserName"
• UserPassword="@UserPassword"
• HttpClientCredentialType="Ntlm|Windows"

The Path attribute is requried, the other attributes are optional. Default values are bold. If Report Server requries authentication UserName and UserPassword attributes values must be assigned. Otherwise (for current windows profile) remove the attributes or leave them as is (the program ignores @masked values). ParameterLanguage value is essential when you provide reports date parameter values. Report server tries to evaluate dates values MM/DD/YYYY (en-us) or DD/MM/YYYY (en-au) taking into account ParameterLanguage.
Test case XML element has Assert attribute which can have difeerent values:

• Check if this field is blank means Assert="IsNotNull"
• Compare cell value from report with static value means Assert="AreEqual" Value="<Static Value>"
• Compare cell value from summary report with detail report total means Assert="AreEqual" Value="<Value>" and <DrillDownReport Value>

The next step is to define Path attribute. This attribute reflects XPath notation, i.e. each SSRS report has unique cell name (auto generated or assigned by developer). For example, in Sales Order Details report there are a number of lines of order. You can retrieve unit price of line with number 1 using the following Path value
//Detail[@LineNumber='1']/@UnitPrice
Here is example of test result:

As you can see, 3 test cases passed successfully and one failed. The reason of this fail is child report has another date range criteria. This issue is described here


How to present and validate test result
Each time when the executable runs it reads input file (Settings.xml) and generates output file. By default, the output file named is "TestSuite yyyy-MM-dd hh-nn-ss.xml". It has the same structure as input file, but for each TestCase element it has additional attribute "Passed" with value True or False. That output file is a target of testing. Also output file has a refference to xsl file. It allows to open file in browser and transform xml data into html. For that reason 2 additional files are used:

• Settings.xsl
• Settings.css

The xml data is presented by browser in readable format like the following:

Also Settings.xsd file was included into solution to validate Settings.xml file. It could be handy when Settings.xml is auto generated and must be checked before using. It provides valuable for continuos integration process as well.


Restrictions

1. Tested with SSRS 2008R2 and SSRS 2012 in Native and SharePoint modes.
2. In SharePoint mode it was tested for 2010 version only
3. Express editions of SSRS does not support XML rendering. The solution is not suatable for Express edition of SSRS. Database engine could be any edition
4. The application is compiled with .Net 4.0 in Visual Studio 2012. But it could be easily converted to .Net 3.5.
5. Application does not require configuration file (*.config). It reads metdata from Settings.xml node attribute <ReportServer Path=...>. It allows dynamically configure endpoint and connect to SSRS SOAP webservices ReportExecution2005.asmx and ReportService2010.asmx. Names of services do not reflect version of SSRS. They are provided by 2008R2 and 2012 in both modes (Native and SharePoint).