Test configuration

Configuration to the Pumpo#5 framework is done mainly thanks to configuration files. The main configuration file must be named exactly config.conf and must be placed exactly on the following path:

src/test/resources/config.conf

Nested configuration

The configuration file can contain:

Key value pairs using the syntax key=value on each line. In case the value is a string, it must be enclosed in double quotes.
Relative path to another configuration file to be included using the syntax include "directory/other-config-file.conf".

Nesting of configuration files is unlimited. All key value pairs are flattened. In case a key is present several times, the value of the last occurrence will be taken.

This pattern can be used in most complex environments where each person is responsible to maintain configuration for a set of systems under test. By splitting the configuration map into many files it is much easier to manage the configuration using any git flow because merge conflicts will be very limited.

Connection to the testing farm

In most cases the most important configuration is the URL of the testing farm. It needs to be configured under the key webdriver.url. For example when using a paid service Browserstack, the configuration would look like:

webdriver.url="https://<user>>:<token>@hub-cloud.browserstack.com/wd/hub"

In case you are running your own farm based on our docker-compose on your computer, then it would look like:

webdriver.url="http://127.0.0.1:4444/wd/hub"

Connection to systems under test

A critical topic is how to manage all configurable values required to connecting to systems under test. Those include mainly:

URLs
Authentication credentials
Timeouts and other non-functional settings The Pumpo#5 library includes a pattern how to manage those settings in configuration files that may or not be included in the versioning system and how to pass the values to the intermediary clients connecting to systems under tests (mainly browsers but also other clients called drivers).

Configuration as session capabilities

One way to configure credentials and other settings when using any browser, emulator or proxy (hereafter called driver) is to pass them as session capabilities using the @Capability annotation as follows:

public class TestClass {
    @Test
    public void doSomething(@Capability(key = "browserName", value = "driverId")
                            @Capability(key = "key1", value = "configKey1", type = ValueType.STRING_PROPERTY)
                                    AbcApplication abcApplication) {
        ...
        abcApplication.step1()
                .step2();
        ...
    }
}

Where:

AbcApplication is the selected class of type Application that represents the system under test
key1 is the key of the setting
configKey1 is the key under which the value of the setting can be found anywhere in the nested configuration files
type is the type of the value, ValueType.STRING_PROPERTY in most cases

Usually those key value pairs include some url and optionally authentication details or non-functional settings such as timeouts or connection modes which is specific to each underlying technical stack.

Unlimited number of capabilities can be specified. Capabilities are serialised as a map and passed to the target driver during session initialisation. In case you need to change capabilities during one test, you need to instantiate another application and pass different capabilities to it.

Conditional capabilities

It is possible to specify a capability that will be used only when certain condition (another capability is present, another capability has certain value) is met. For an example check out Conditional capabilities section of Mobile DSL reference.

Configuration as method arguments

Another way to configure connection to systems under test is to pass values as arguments in the methods that are specific for each client (domain). The main example is how URLs are defined for the WEB domain or how endpoints are defined for the REST/SOAP/HTTP domain, see below.

General settings

Pumpo#5 supports following settings (via nested config files):

Key	Type	Description
pn5.webdriver.tls.ignore	boolean	Disabled TLS certificates check on farm connections for all applications

Following capabilities can be applied to any application:

Capability	Type	Description
pn5:tls:ignore	boolean	Setting it to 'true' will allow you to ignore issues with TLS certificates on your test farm that is running over https.

Session settings

Session settings can be passed to the test farm using the @Capability annotation of whatever proxy host (browser or any other driver).

Capability	Type	Description
sessionTimeout	String	Timeout until the session is killed if there is no activity. It is specified in seconds, e.g. "120s". Default value is "70s".
name	String	Name of the session that can be used e.g. to specify the name of the tester or of the test itself
enableVNC	boolean	Enable VNC on the test farm UI - makes sense only for browsers
pn5:manualSession	boolean	Whether a session should be handled manually
pn5:keepSession	boolean	Whether the session should be kept open event if test successful or aborted
pn5:lookup:default	String	Which type should be used as default for element lookup
pn5:remoteTimeout	Integer	Timeout, in seconds, to wait for a response from an external service. Default value is "70".
pn5.capabilities.dump	String	Log current capabilities to console

We do not reproduce documentation of standard capabilities supported by Selenoid, only those that are key based on our experience. Please see an exhaustive list in the reference documentation

pn5:manualSession

When set to true session opening will not be handled automatically and one needs to call the method sessionStart(). Later on the methods sessionClose() and again sessionStart() can be called to close the session and reopen it later. This enables to start the session just in time or to close it immediately after using the driver even if the test still continues. This enables long-running tests to not consume resources on the test farm while not necessary. Note: sessions are automatically closed if the test is successful or aborted. To keep the session open see the other session related annotation driver8:keepSession below.

Note: Even if this annotation is an extension of the protocol it will also work with commercial test farms because session is closed from the client side.

pn5:keepSession

When set to true the session will not be closed even if the test is successful or aborted. By default, any session is immediately closed after the test is successful or aborted. This helps to keep resource consumption as low as possible. In case of test failure the configured (or default) session timeout will be used by the test farm even if the client (test) has disconnected.

Note: Even if this annotation is an extension of the protocol it will also work with commercial test farms because session is closed from the client side.

pn5:lookup:default

What lookup type should be used as default. This value should be remembered and used for all annotations where "by" is set to DEFAULT.

Valid values are "id", "xpath", "name", "text", "className", "description", "automatorCommand", "method" and "default" (which is application dependent).

Please, bear in mind that not all the lookup types are valid and available, depending on which application type you are testing.

pn5:remoteTimeout

Timeout, in seconds, to wait idle for a response from an HTTP request to an external endpoint or service. After this timeout, the channel will close, and you will see a TimeoutException in the test results.

Default value is 70 (e.g. 70 seconds). It has to be less than or equal to sessionTimeout value (which by default is also 70 seconds).

pn5.capabilities.dump

When set to true, all the capabilities used by the current session are logged to the console. Default value is true.

WEB

Websites are tested via browsers that run remotely on a test farm. To use such remote browser in a test method the test method needs to be passed as argument a WebApplication annotated with following capabilities:

Key	Type	Mandatory	Description
browserName	N/A	Mandatory	Value needs to match the test farm configuration.

Configuration of web urls and credentials for websites / web applications is defined in the page objects representing the application. A page object looks typically as:

@Navigate("https://www.google.com")
public interface SearchPage extends WebDriverAccessor {

    @SetValue(value = "q", by = Lookup.NAME)
    SearchPage typeIntoSearchBox(String term);

    @ExtendedAction
    default SearchPage submit() {
        getDriver().findElement(By.name("q")).submit();
        return null;
    }

    SearchResultsPage onSearchResultsPage();
}

The annotation @Navigate("https://www.google.com") contains the configuration of where to find the SearchPage.

HTTP

To use a remote HTTP driver in a test method the test method needs to be passed as argument a HttpApplication annotated with following capabilities:

Key	Type	Mandatory	Description
browserName	N/A	Mandatory	Value needs to match the test farm configuration.

browserName

This capability allows routing commands to the right driver. The value needs to be inline with what is configured on your test farm infrastructure. Our default is to use pn5-driver8 for all areas covered by our universal component Driver8.

Endpoint

Configuration of endpoints to accessed is passed using one of the domain specific methods prepareRestRequest, prepareSoapRequest or prepareHttpRequest.

Examples

The following test will connect to the REST endpoint https://foobar.com/api/dogs using the HTTP GET method and assert that the response code is 200.

public class Test {
    @Test
    void listDogs(@Capability(key = "browserName", value = "pn5-driver8")
                                HttpApplication restClient) {
        restClient
                .prepareRestRequest("https://foobar.com/api/dogs","GET")
                .sendAndGetResponse()
                .assertStatus(200);
    }
}

SQL

To use a remote SQL driver in a test method the test method needs to be passed as argument a SqlApplication annotated with following capabilities:

Key	Type	Mandatory	Description
browserName	N/A	Mandatory	Value needs to match the test farm configuration.
pn5:queryTimeout	Integer	Optional	Timeout in seconds waiting for a query to return.
pn5:sqlDialect	String	Mandatory	The dialect of the database server
pn5:sqlHostname	String	Mandatory	The hostname of the database server
pn5:sqlPort	Integer	Mandatory	The port of the database server
pn5:sqlDatabase	String	Mandatory	The name of the database / schema
pn5:sqlUsername	String	Optional	Username to authenticate with
pn5:sqlPassword	String	Optional	Password to authenticate with
pn5:hidden:sqlPassword	String	Optional	Same as above, but the value will be hidden and protected in all logs.

browserName

This capability allows routing commands to the right driver. The value needs to be inline with what is configured on your test farm infrastructure. Our default value is pn5-driver8 for all areas covered by our universal component Driver8.

pn5:queryTimeout

This capability allows setting a timeout, in seconds, to wait for a query or update statement being executed to return a result. The default value is 60 seconds.

pn5:sqlDialect

The dialect of the database server to connect to. Possible values are sqlServer, postgresql, mysql and mariadb

pn5:sqlHostname

The hostname of the database server to connect to. Can be a fully qualified domain name or IP address.

pn5:sqlPort

The port on which the database server is listening. Should be an integer.

pn5:sqlUsername

The username to use when authenticating with the database server.

pn5:sqlPassword / pn5:hidden:sqlPassword

The password to use when authenticating with the database server.

Examples

The following test will connect to a Microsoft SQL Server on hostname and port to be found in configuration files under keys sql.host and sql.port. The schema is configured under key sql.db. Dialect is configured under key sql.dialect. Authentication will use the user and password to be found in configuration files under keys sql.user and sql.pass. Query timeout is set to 5 seconds.

public class Test {
  @Test
  void listRows(@Capability(key = "browserName", value = "pn5-driver8")
                @Capability(key = "pn5:sqlDialect", value = "sql.dialect", type = ValueType.STRING_PROPERTY)
                @Capability(key = "pn5:sqlHostname", value = "sql.host", type = ValueType.STRING_PROPERTY)
                @Capability(key = "pn5:sqlPort", value = "sql.port", type = ValueType.INTEGER_PROPERTY)
                @Capability(key = "pn5:sqlDatabase", value = "sql.db", type = ValueType.STRING_PROPERTY)
                @Capability(key = "pn5:sqlUsername", value = "sql.user", type = ValueType.STRING_PROPERTY)
                @Capability(key = "pn5:sqlPassword", value = "sql.pass", type = ValueType.STRING_PROPERTY)
                @Capability(key = "pn5:queryTimeout", value = "5", type = ValueType.INTEGER)
                        SqlApplication sqlClient) { 
      client
              .prepareQuery("select count(*) FROM [schemaName].[tableName] WHERE [name] = '#{name}'")
              .withParam("name", "John Doe")
              .execute();
  }
}

MongoDB

To use a remote MongoDB driver, you need to pass an object that implements MongoApplication interface, annotated with following capabilities:

Key	Type	Mandatory	Description
browserName	N/A	Mandatory	Value needs to match the test farm configuration.
pn5:mongoHostname	String	Mandatory	The hostname of the database server
pn5:mongoPort	Integer	Mandatory	The port of the database server
pn5:mongoDatabase	String	Mandatory	The name of the database to connect to
pn5:mongoUsername	String	Mandatory	Username to authenticate with
pn5:mongoPassword	String	Mandatory	Password to authenticate with

browserName

pn5:mongoHostname

The hostname of the database server to connect to. Can be a fully qualified domain name or IP address.

pn5:mongoPort

The port on which the database server is listening. Should be an integer.

pn5:mongoUsername

The username to use when authenticating with the database server.

pn5:mongoPassword

The password to use when authenticating with the database server.

Examples

The following test will connect to a MongoDB on hostname and port to be found in configuration files under keys mongo.host and mongo.port, and database configured under key mongo.database. Authentication will use the user and password to be found in configuration files under keys mongo.user and mongo.password.

It will then prepare a command to create a new collection, execute and assert the command was successful. Finally, it will prepare another command to list all the user collections on the database, execute, and assert the collection was really created and it exists.

class TestClass {

    @Test
    void createAndDropCollection(
            @Capability(key = BROWSER_NAME, value = BROWSER_NAME_MONGO)
            @Capability(key = CAPABILITY_MONGO_HOSTNAME, value = "mongo.host", type = ValueType.STRING_PROPERTY)
            @Capability(key = CAPABILITY_MONGO_PORT, value = "mongo.port", type = ValueType.INTEGER_PROPERTY)
            @Capability(key = CAPABILITY_MONGO_DATABASE, value = "mongo.database", type = ValueType.STRING_PROPERTY)
            @Capability(key = CAPABILITY_MONGO_USERNAME, value = "mongo.user", type = ValueType.STRING_PROPERTY)
            @Capability(key = CAPABILITY_MONGO_PASSWORD, value = "mongo.password", type = ValueType.STRING_PROPERTY)
            MongoApplication mongoApplication) {

        mongoApplication
                .prepareCommand()
                .createCollection(NEW_COLLECTION)
                .execute()
                .assertIsSuccessful()
                .andThen()
                .prepareCommand()
                .listCollections()
                .execute()
                .assertCollectionIsPresent(NEW_COLLECTION);
    }
}

SFTP

To use a remote SFTP driver in a test method the test method needs to be passed as argument a SftpApplication annotated with following capabilities:

Key	Type	Mandatory	Description
browserName	N/A	Mandatory	Value needs to match the test farm configuration.
pn5:sftpHost	String	Mandatory	The SFTP server host to connect to.
pn5:sftpPort	String	Mandatory	The SFTP server port to connect to.
pn5:sftpUsername	String	Mandatory	The username to authenticate with against the SFTP server.
pn5:sftpPassword	String	Mandatory	The password to authenticate with against the SFTP server.
pn5:hidden:sftpPassword	String	Mandatory	Same as above, but the value will be hidden and protected in all logs.

browserName

This capability allows routing commands to the right driver. The value needs to be inline with what is configured on your test farm infrastructure. Our default is to use pn5-sftp for all areas covered by our universal component Driver8.

pn5:sftpHost

The host of the SFTP server. Can be a fully qualified domain name or IP address. No protocol qualifier should be specified.

pn5:sftpPort

The port on which the SFTP server is listening. Beware that communication on the standard port 22 is usually not allowed by default in enterprise environments.

pn5:sftpUsername

The user to authenticate with against the SFTP server.

pn5:sftpPassword / pn5:hidden:sftpPassword

The password to authenticate with against the SFTP server.

Examples

The following test will connect to a remote SFTP server running on the host and port found in the configuration file, under keys sftp.host and sftp.port. The authentication is by username and password, with values found under keys sftp.user and sftp.pass.

It will upload a CSV file found in local path path/to/file/forUpload.csv to remote folder in server /remoteFolder.

public class TestClass {
    @Test
    public void uploadCsvFileTest(
            @Capability(key = "browserName", value = "pn5-sftp")
            @Capability(key = "pn5:sftpHost", value = "sftp.host", type = STRING_PROPERTY)
            @Capability(key = "pn5:sftpPort", value = "sftp.port", type = INTEGER_PROPERTY)
            @Capability(key = "pn5:sftpUser", value = "sftp.user", type = STRING_PROPERTY)
            @Capability(key = "pn5:sftpPassword", value = "sftp.pass", type = STRING_PROPERTY)
                    SftpApplication sftp) {

        sftpClient.upload("path/to/file/forUpload.csv", "/remoteFolder");
    }
}

KAFKA

To use a remote Kafka driver in a test method the test method needs to be passed as argument a KafkaApplication annotated with following capabilities:

Key	Type	Mandatory	Description
browserName	N/A	Mandatory	Value needs to match the test farm configuration.
pn5:kafkaUrl	String	Mandatory	The URL in the form `host:port`, without any protocol qualifier.
pn5:kafkaUsername	String	Optional	The username to use when authentication is enabled. In case Kafka accepts anonymous access, the parameter must not be passed.
pn5:kafkaPassword	String	Optional	The password to use when username has been specified. In case username was not specified, password will be ignored.
pn5:hidden:kafkaPassword	String	Optional	Same as above, but the value will be hidden and protected in all logs.
pn5:kafkaGroupId	String	Optional	The consumer group id to be used in case a read query is processed.
pn5:kafkaProtocol	String	Optional	The value of this determines the security protocol used for communication between clients and brokers.

browserName

This capability allows routing commands to the right driver. The value needs to be inline with what is configured on your test farm infrastructure. Our default is to use pn5-kafka for all areas covered by our universal component Driver8.

pn5:kafkaUrl

Kafka url can be specified using a hostname, or an IP address and a port in the form host:port. In case kafka uses authentication over SSL the certificate provided by Kafka during the handshake must match the host specified in the configuration.

In case you are using a certificate authority that is not trusted by standard JDK distributions then the driver needs to be enhanced with the certification authority of your choice.

pn5:kafkaUsername + pn5:kafkaPassword / #### pn5:kafkaUsername + pn5:hidden:kafkaPassword

Kafka username and password are used both in authentication using SASL_PLAINTEXT or SASL_SSL with PLAIN mechanism.

pn5:kafkaProtocol

Kafka protocol determines the security protocol used for communication between clients and brokers. The value of this configuration option defines the type of encryption and authentication mechanisms employed in the Kafka cluster. The values you can currently specify are SASL_SSL and SASL_PLAINTEXT. If you don't specify any, SASL_SSL will be chosen by default.

pn5:kafkaGroupId

The consumer group id to be used in case a read query is processed. If the parameter is omitted then manual assignment will be used and this is probably the preferred way to work with Kafka in tests.

Examples

The following test will connect to Kafka on the url that needs to be found in configuration files under the key kafka.url using no authentication (anonymous) access.

public class TestClass {
    @Test
    public void topicListTest(@Capability(key = "browserName", value = "pn5-driver8")
                              @Capability(key = "pn5:kafkaUrl", value = "kafka.url", type = ValueType.STRING_PROPERTY)
                                      KafkaApplication kafka) {
        kafka
                .listTopics()
                .assertThatContainsTopicNamed("shouldExist");
    }
}

The following test will connect to Kafka on the url that needs to be found in configuration files under the key kafka.url using SASL_SSL with PLAIN mechanism authentication with username and password to be found in config files under respective keys kafka.username and kafka.password. In case a read query is to be processed the groupId "testGroup" will be used.

public class TestClass {
    @Test
    public void topicListTest(@Capability(key = "browserName", value = "pn5-driver8")
                              @Capability(key = "pn5:kafkaUrl", value = "kafka.url", type = ValueType.STRING_PROPERTY)
                              @Capability(key = "pn5:kafkaUsername", value = "kafka.username", type = ValueType.STRING_PROPERTY)
                              @Capability(key = "pn5:kafkaPassword", value = "kafka.password", type = ValueType.STRING_PROPERTY)
                              @Capability(key = "pn5:kafkaGroupId", value = "testGroup", type = ValueType.STRING_PROPERTY)
                                      KafkaApplication kafka) {
        kafka
                .listTopics()
                .assertThatDoesNotContainTopicNamed("shouldNotExist");
    }
}

Samba

To use a remote Samba driver in a test, the test method needs a SambaApplication parameter annotated with following capabilities:

Key	Type	Mandatory	Description
browserName	N/A	Mandatory	Value needs to match the test farm configuration.
pn5:sambaHost	String	Mandatory	The Samba server host to connect to.
pn5:sambaPort	String	Mandatory	The Samba server port to connect to.
pn5:sambaUsername	String	Mandatory	The username to authenticate with against the Samba server.
pn5:sambaPassword	String	Mandatory	The password to authenticate with against the Samba server.
pn5:hidden:sambaPassword	String	Mandatory	Same as above, but the value will be hidden and protected in all logs.
pn5:sambaDomain	String	Mandatory	The domain to connect in the Samba server
pn5:sambaSharename	String	Mandatory	The name of the share to connect in the Samba server

browserName

This capability allows routing commands to the right driver. The value needs to be inline with what is configured on your test farm infrastructure. Our default is to use pn5-samba for all areas covered by our universal component Driver8.

pn5:sambaHost

The host of the Samba server. Can be a fully qualified domain name or IP address. No protocol qualifier should be specified.

pn5:sambaPort

The port the Samba server is listening to. Beware that communication on the standard port 445 is usually not allowed by default in enterprise environments.

pn5:sambaUsername

The user to authenticate with against the Samba server.

pn5:sambaPassword / pn5:hidden:sambaPassword

The password to authenticate with against the Samba server.

pn5:sambaDomain

The domain to connect to in the Samba server.

pn5:sambaSharename

The name of share where you want to connect to in the Samba server.

Examples

The following test will connect to a Samba server on hostname and port to be found in configuration files under keys samba.host and samba.port, use the domain configured under key samba.domain, and the share name configured under key samba.shareName. Authentication will use the user and password to be found in configuration files under keys samba.username and samba.password.

It will then list the files in the root directory and assert that it contains the default paths . (for current directory) and .. (for previous directory) in the listed folders and files returned.

Finally, it will print information for all the folders and files found in the directory.

public class TestClass {
    @Test
    public void listFilesInCurrentEmptyDirectory(@Capability(key = "browserName", value = "pn5-samba")
                                                 @Capability(key = "pn5:sambaHost", value = "samba.host", type = ValueType.STRING_PROPERTY)
                                                 @Capability(key = "pn5:sambaPort", value = "samba.port", type = ValueType.STRING_PROPERTY)
                                                 @Capability(key = "pn5:sambaUsername", value = "samba.username", type = ValueType.STRING_PROPERTY)
                                                 @Capability(key = "pn5:sambaPassword", value = "samba.password", type = ValueType.STRING_PROPERTY)
                                                 @Capability(key = "pn5:sambaDomain", value = "samba.domain", type = ValueType.STRING_PROPERTY)
                                                 @Capability(key = "pn5:sambaSharename", value = "samba.shareName", type = ValueType.STRING_PROPERTY)
                                                         SambaApplication samba) {
        samba
                .listFiles("")
                .assertThatContainsFileOrFolderNamed(".")
                .assertThatContainsFileOrFolderNamed("..")
                .printFilesInformation();
    }
}

Test configuration

Nested configuration​

Connection to the testing farm​

Connection to systems under test​

Configuration as session capabilities​

Conditional capabilities​

Configuration as method arguments​

General settings​

Session settings​

pn5:manualSession​

pn5:keepSession​

pn5:lookup:default​

pn5:remoteTimeout​

pn5.capabilities.dump​

WEB​

HTTP​

browserName​

Endpoint​

Examples​

SQL​

browserName​

pn5:queryTimeout​

pn5:sqlDialect​

pn5:sqlHostname​

pn5:sqlPort​

pn5:sqlUsername​

pn5:sqlPassword / pn5:hidden:sqlPassword​

Examples​

MongoDB​

browserName​

pn5:mongoHostname​

pn5:mongoPort​

pn5:mongoUsername​

pn5:mongoPassword​

Examples​

SFTP​

browserName​

pn5:sftpHost​

pn5:sftpPort​

pn5:sftpUsername​

pn5:sftpPassword / pn5:hidden:sftpPassword​

Examples​

KAFKA​

browserName​

pn5:kafkaUrl​

pn5:kafkaUsername + pn5:kafkaPassword / #### pn5:kafkaUsername + pn5:hidden:kafkaPassword​

pn5:kafkaProtocol​

pn5:kafkaGroupId​

Examples​

Samba​

browserName​

pn5:sambaHost​

pn5:sambaPort​

pn5:sambaUsername​

pn5:sambaPassword / pn5:hidden:sambaPassword​

pn5:sambaDomain​

pn5:sambaSharename​

Examples​

Nested configuration

Connection to the testing farm

Connection to systems under test

Configuration as session capabilities

Conditional capabilities

Configuration as method arguments

General settings

Session settings

pn5:manualSession

pn5:keepSession

pn5:lookup:default

pn5:remoteTimeout

pn5.capabilities.dump

WEB

HTTP

browserName

Endpoint

Examples

SQL

browserName

pn5:queryTimeout

pn5:sqlDialect

pn5:sqlHostname

pn5:sqlPort

pn5:sqlUsername

pn5:sqlPassword / pn5:hidden:sqlPassword

Examples

MongoDB

browserName

pn5:mongoHostname

pn5:mongoPort

pn5:mongoUsername

pn5:mongoPassword

Examples

SFTP

browserName

pn5:sftpHost

pn5:sftpPort

pn5:sftpUsername

pn5:sftpPassword / pn5:hidden:sftpPassword

Examples

KAFKA

browserName

pn5:kafkaUrl

pn5:kafkaUsername + pn5:kafkaPassword / #### pn5:kafkaUsername + pn5:hidden:kafkaPassword

pn5:kafkaProtocol

pn5:kafkaGroupId

Examples

Samba

browserName

pn5:sambaHost

pn5:sambaPort

pn5:sambaUsername

pn5:sambaPassword / pn5:hidden:sambaPassword

pn5:sambaDomain

pn5:sambaSharename

Examples