API

Classes

Environment

It manages the use of environment variables by providing useful methods for checking and obtaining the variables.

Process

Provides Puppeteer initialization by creating Browser and Page instances to provide the browser context necessary for the execution of a custom function.

Environment

It manages the use of environment variables by providing useful methods for checking and obtaining the variables.

Kind: global class

environment.PRODUCTION

Value of the variable defined by the NodeJS processor to identify the Production execution environment.

Kind: instance property of Environment

environment.IMPRESSIONIST_VERSION

The current version of the library. It can be used, for example, for bug reporting plugins.

Kind: instance property of Environment

Environment.is(environment) ⇒ boolean

Check if a specific environment is running. i.e prod or dev.

Kind: static method of Environment Returns: boolean - Result true or false.

ParamTypeDescription

environment

string

Identifier of the ENV environment variable. For example, prod, dev, etc.

Example

if(Environment.is(Environment.PRODUCTION)) {
     ...
}

Environment.has(variable) ⇒ boolean

Check if a specific environment variable exists.

Kind: static method of Environment Returns: boolean - Result true or false.

ParamTypeDescription

variable

string

The variable to check.

Example

if(Environment.has('SENTRY_TAGS')) {
     ...
}

Environment.get(variable) ⇒ string | Array | object | null

Return a specific env var. If not exist return null.

Kind: static method of Environment

ParamTypeDescription

variable

string

The variable to extract.

Example

for(const [name, value] of Object.entries(Environment.get('SENTRY_TAGS'))) {
     sentry.setTag(name, value);
}

Process

Provides Puppeteer initialization by creating Browser and Page instances to provide the browser context necessary for the execution of a custom function.

Kind: global class Summary: Initialize Puppeteer.

process.browserController

Provides methods and features to interact with the browser. By default, PuppeteerController.

Kind: instance property of Process

Process.execute(url, customFunction, options) ⇒ Promise

It provides the necessary context to run a function in the puppeteer or browser context by executing a series of steps, such as initializing a Browser instance and applying extra configurations defined by an input parameter, initializing a Page instance and applying default configurations to it so that the Impressionist library can be used in the browser context of that specific instance.

Kind: static method of Process Returns: Promise - Promise object that represents the result of the custom function.

ParamTypeDescription

url

string

Target URL for scraping process.

customFunction

function

Custom function to be executed in the Puppeteer context. Like customFunction(browser, page, ...args) { ... }.

options

Object

Options to configure the browser and page. Please check the browser configuration in https://pptr.dev/api/puppeteer.puppeteerlaunchoptions Please check the page navigation options in https://pptr.dev/api/puppeteer.page.goto.

Example (Basic Usage)

(async () => {
    const data = await Impressionist.Process.execute(url, scrape);
    console.log(JSON.stringify(data));
})(scrape);

async function scrape(browser, page) {
     ...
}

Example (Enabling Browser User Interface)

(async () => {
     return await Impressionist.Process.execute(
         url,
         function scrape() { ... },
         {
             browserOptions: { headless: false }
         }
     );
})();

Example (Set a specific page navigation timeout)

(async () => {
     return await Impressionist.Process.execute(
         url,
         function scrape() { ... },
         { 
             pageOptions: { timeout: 120000 },
         },
     );
})();

Process.initialize(url, [browserOptions]) ⇒ Promise.<symbol>

Perform actions to initialize a connection to a page using the browser controller.

Kind: static method of Process Returns: Promise.<symbol> - Promise which resolves to a connection identifier.

ParamTypeDefaultDescription

url

string

URL or browser Endpoint.

[browserOptions]

object

{}

Options to configure the browser controller.

Process.configureConnection(connectionIdentifier)

Configure a page connection.

Kind: static method of Process

ParamTypeDescription

connectionIdentifier

symbol

Unique identifier for a page connection.

Process.enableImpressionistFeatures(connectionIdentifier)

Enable all the Impressionist features to be used in the browser context.

Kind: static method of Process

ParamTypeDescription

connectionIdentifier

symbol

Unique identifier for a page connection.

Process.executeFunction(connectionIdentifier, customFunction) ⇒ Promise.<any>

Execute the function in the browser context provided by the browser controller.

Kind: static method of Process Returns: Promise.<any> - Promise which resolves to the result of the execution of the function.

ParamTypeDescription

connectionIdentifier

symbol

Unique identifier for a page connection.

customFunction

function

Custom function to be executed in the Puppeteer context. Like customFunction(browser, page, ...args) { ... }.

Process.handleError(error, connectionIdentifier) ⇒ Promise.<any>

Handle errors.

Kind: static method of Process Returns: Promise.<any> - Promise which resolves to any result of error handling.

ParamTypeDescription

error

Error

Error object.

connectionIdentifier

symbol

Unique identifier for a page connection.

Process.close(connectionIdentifier) ⇒ Promise.<any>

Close connections.

Kind: static method of Process Returns: Promise.<any> - Promise which resolves to any result of closing connections.

ParamTypeDescription

connectionIdentifier

symbol

Unique identifier for a page connection.

Process.exposeLogger(connectionIdentifier)

Exposes the logger function to be used in the browser context.

Kind: static method of Process

ParamTypeDescription

connectionIdentifier

symbol

Unique identifier for a page connection.

Process.enableCollector(connectionIdentifier)

Load the library classes in the browser environment.

Kind: static method of Process

ParamTypeDescription

connectionIdentifier

symbol

Unique identifier for a page connection.

Process.registerSelectors(connectionIdentifier)

Make the registration of Selectable sub-classes using their static method register.

Kind: static method of Process

ParamTypeDescription

connectionIdentifier

symbol

Unique identifier for a page connection.

Process.registerStrategies(connectionIdentifier)

Register strategies.

Kind: static method of Process

ParamTypeDescription

connectionIdentifier

symbol

Unique identifier for a page connection.

Process.addProxyFunctions(connectionIdentifier)

Add functions to be used in Browser Context.

Kind: static method of Process

ParamTypeDescription

connectionIdentifier

symbol

Unique identifier for a page connection.

Process.connect(browserWSEndpoint, customFunction, [...args]) ⇒ Promise.<any>

Execute a function in a specific browser endpoint.

Kind: static method of Process Returns: Promise.<any> - Promise object that represents the result of the custom function.

ParamTypeDefaultDescription

browserWSEndpoint

string

customFunction

function

Custom function to be executed in the Puppeteer context. Like customFunction(browser, page, ...args) { ... }.

[...args]

Array.<any>

[]

Any parameter necessary for the custom function.

Process.setBrowserController(browserController)

Set a browser controller which Impressionist use to interact with the browser context.

Kind: static method of Process

ParamTypeDescription

browserController

object

Browser controller.

Process.loadPlugin(plugin)

Load a plugin.

Kind: static method of Process

ParamTypeDescription

plugin

object

A class to extends or modify the Impressionist behavior.

Classes

Puppeteer

Controls access to puppeteer methods and features.

PuppeteerController

Provides shorcut methods to Puppeteer configurations for initialize

MonitorManager

Log on each of the subscribed monitoring tools.

Pino

Initialize Pino and expose its functionality for login.

Sentry

Provides an interface for Sentry integration with Puppeteerist.

Functions

Sentry.()

Add the necessary tags for Sentry. Some come from environment variables and others are defined by the library locally.

Puppeteer

Controls access to puppeteer methods and features.

Kind: global class

Puppeteer.launch([options]) ⇒ Promise.<object>

Enables a connection to control the browser.

Kind: static method of Puppeteer Returns: Promise.<object> - Promise which resolves to browser instance.

ParamTypeDefaultDescription

[options]

object

{}

Please read the documentation about the Launch Options.

Puppeteer.newPage(browser) ⇒ Promise.<object>

Create a new Page object.

Kind: static method of Puppeteer Returns: Promise.<object> - Promise which resolves to a new Page object. The Page is created in a default browser context.

ParamTypeDescription

browser

object

Browser instance.

Puppeteer.close(...controllers)

Close any instance of Page or Browser.

Kind: static method of Puppeteer

ParamTypeDescription

...controllers

any

Page or Browser instances.

Puppeteer.goto(page, url)

Navigate to a specific URL.

Kind: static method of Puppeteer

ParamTypeDescription

page

object

Page instance.

url

string

URL.

Puppeteer.evaluate(page, pageFunction, ...args) ⇒ Promise.<any>

Execute a function in the browser context.

Kind: static method of Puppeteer Returns: Promise.<any> - Promise which resolves to a result of the function executed in the browser context.

ParamTypeDescription

page

object

Page instance.

pageFunction

function

A function to be executed in the browser context.

...args

any

Function arguments.

Puppeteer.exposeFunction(page, name, puppeteerFunction)

Expose a NodeJS function to be called from the browser context.

Kind: static method of Puppeteer

ParamTypeDescription

page

object

Page Instance.

name

string

Function name to be called from the browser context.

puppeteerFunction

function

Function that is going to be executed in the NodeJS context.

Puppeteer.addScriptTag(page, options)

Add a new script tag in the HTML layout.

Kind: static method of Puppeteer

ParamTypeDescription

page

object

Page instance.

options

object

Options for load content in the script tag. Please check the documentation.

Puppeteer.addEventListener(page, method, event, handler)

Add an event listener to page.

Kind: static method of Puppeteer

ParamTypeDescription

page

object

Page instance.

method

string

EventEmitter method.

event

string | symbol

The event to add the handler to.

handler

function

The event listener that will be added.

PuppeteerController

Provides shorcut methods to Puppeteer configurations for initialize

Kind: global class

puppeteerController.browser

Browser instance.

Kind: instance property of PuppeteerController

puppeteerController.pages

Page instances.

Kind: instance property of PuppeteerController

PuppeteerController.initialize(url, [options]) ⇒ Promise.<symbol>

Open a connection to an URL using a page instance.

Kind: static method of PuppeteerController Returns: Promise.<symbol> - Promise which resolves to a unique identifier represented by a Symbol.

ParamTypeDescription

url

string

URL.

[options]

object

Please read the documentation about the Launch Options.

PuppeteerController.close([identifier])

Close connections.

Kind: static method of PuppeteerController

ParamTypeDescription

[identifier]

symbol

Unique identifier for a page connection.

PuppeteerController.evaluate(identifier, pageFunction, ...args) ⇒ Promise.<any>

Evaluate a function in a specific page.

Kind: static method of PuppeteerController Returns: Promise.<any> - Promise which resolves to the result of the pageFunction.

ParamTypeDescription

identifier

symbol

Unique identifier for a page connection.

pageFunction

function

A function to be evaluated in the browser context.

...args

any

Arguments for being passed to the pageFunction.

PuppeteerController.execute(identifier, puppeteerFunction) ⇒ Promise.<any>

Execute a function which provides browser and page as parameters.

Kind: static method of PuppeteerController Returns: Promise.<any> - Promise which resolves to the result of the puppeteerFunction.

ParamTypeDescription

identifier

symbol

Unique identifier for a page connection.

puppeteerFunction

function

Custom function.

PuppeteerController.inject(identifier, functionality)

Inject a function in the HTML layout.

Kind: static method of PuppeteerController

ParamTypeDescription

identifier

symbol

Unique identifier for a page connection.

functionality

function

Function to be load as a script tag in the page.

PuppeteerController.expose(identifier, functionality, [name])

Expose a NodeJS function to be used from the browser context.

Kind: static method of PuppeteerController

ParamTypeDefaultDescription

identifier

symbol

Unique identifier for a page connection.

functionality

function

Function to be exposed.

[name]

string

"functionality.name"

Function name to be used in the browserContext.

PuppeteerController.enableProxy(identifier, proxy)

Setting proxies per page basis.

Kind: static method of PuppeteerController

ParamTypeDescription

identifier

symbol

Unique identifier for a page connection.

proxy

string | object

Proxy to use in the current page. Must begin with a protocol e.g. http://, https://, socks://.

PuppeteerController.enableDebugMode(identifier)

Display console.log messages in environments different than Production.

Kind: static method of PuppeteerController

ParamTypeDescription

identifier

symbol

Unique identifier for a page connection.

MonitorManager

Log on each of the subscribed monitoring tools.

Kind: global class

MonitorManager.log(report)

Register a report.

Kind: static method of MonitorManager

ParamTypeDescription

report

object

Information to be logged in.

MonitorManager.subscribe(logger)

Subscribe to a monitoring or logging tool.

Kind: static method of MonitorManager

ParamTypeDescription

logger

object

Monitoring or logging tool.

MonitorManager.unsubscribe(logger)

Unsubscribe to a monitoring or logging tool.

Kind: static method of MonitorManager

ParamTypeDescription

logger

object

Monitoring or logging tool.

MonitorManager.clear()

Delete all logers. It can be used to discard the default loggers.

Kind: static method of MonitorManager

Pino

Initialize Pino and expose its functionality for login.

Kind: global class

Pino.log(report)

Log a report.

Kind: static method of Pino

ParamTypeDescription

report

object

Information that will be used to compose the report.

Sentry

Provides an interface for Sentry integration with Puppeteerist.

Kind: global class

Sentry.initialize(options, tags)

Specifies the basic options for Sentry initialization.

Kind: static method of Sentry

ParamTypeDescription

options

object

Please check Sentry documentation.

tags

object

Set of tags.

Sentry.log(report)

Log a report.

Kind: static method of Sentry

ParamTypeDescription

report

object

Information that will be used to compose the report.

Sentry.sendException(error) ⇒ Promise.<void>

Send error generated to Sentry while Puppeteer execution.

Kind: static method of Sentry Returns: Promise.<void> - Promise object that represents end of the Sentry actions.

ParamTypeDescription

error

Error

Object that represents the error generated during the execution of the scraper.

Sentry.()

Add the necessary tags for Sentry. Some come from environment variables and others are defined by the library locally.

Kind: global function

Classes

Collector

Collect items from a Collection by being iterated through a context element.

Context

It provides a data structure to control the context, which is an object with two properties. The first is document, which refers to the document object present in the browser context. If the context is not running in Browser context then document is set to null. The other property is element, which stores an element that will be used by some instance during the execution of the collectable tree.

SelectorError

Custom error to be used by the QuerySelector class in case an element or series of elements does not exist in the DOM.

CustomError

Custom error to be used by the Collectable class to identify if the error was created by the error or require methods of the same class.

CollectableError

Custom error to be used by the Collectable class to return any error caused in the execution of the Collectable tree and that does not have a Collectable default method specified.

Logger

Log useful information and register errors.

StrategyManager

Provides a common method to share across the different strategies managers.

TypeValidator

Provides a static method that checks the data type of an incoming value.

LazyLoadHandler

Loads all DOM elements that are handled by a LazyLoad.

Pagination

Handles the pagination of an HTML section.

Functions

Logger.(elements)Array.<object>

Extract useful information from elements and give them a specific format inside a list.

clickAndWait(elementSelector, waitFor)

Click an element in the DOM and then wait for another event like a timeout, a selector being loading in the DOM or wait for the completion of an asyncronous function.

Collector

Collect items from a Collection by being iterated through a context element.

Kind: global class

new Collector(contextAccessor, collection)

ParamTypeDescription

contextAccessor

object

Objects that returns an iterable object as a result of its execution.

collection

Collection

A collection instance.

collector.iterationQuery

Queries to append in iterate method.

Kind: instance property of Collector

collector.call(context) ⇒ Promise.<Array>

Execute the collector.

Kind: instance method of Collector Returns: Promise.<Array> - An object that represents a promise for the collected items.

ParamTypeDescription

context

Object that represents the context or the element that is being passed on nested queries or instances executions.

Example

        await page.evaluate(async () => {
    
            const data = ( function () {

                const css = SelectorDirectory.get('css');

                return new Collector(
                    new Collection({
                        name: css('h1').property('innerText').single()
                    })
                );
                
            } )();
        
        const context = new Context();
        console.log(await data.call(context)); // [{ name: 'Plato Plugin' }]
    });

collector.iterate(queries) ⇒ Collector

Creates a new Collector that have a custom accessor and a collection.

Kind: instance method of Collector Returns: Collector - A Collector instance.

ParamTypeDescription

queries

object

A set of queries or callable objects.

Example

        await page.evaluate(async () => {

            const data = ( function () {

                const css = SelectorDirectory.get('css');
    
                return new ElementCollectorFactory('{#reviews > ul > li}*').iterate({
                    author: '#review-author',
                    title: '#review-title',
                    rating: '#review-rating',
                    body: '#review-body',
                    date: '#review-date'
                });
        
            } )();

            const context = new Context();
            console.log(await data.call(context)); // [{ author: 'John Doe', title: 'It is okay', rating: '4', body: 'Nice product. I would recommend the version X.', date: '01-12-2021' }, { author: 'Richard Roe', title: 'Amazing!', rating: '5', body: 'Really good product.', date: '10-12-2021' }]
        });

Context

It provides a data structure to control the context, which is an object with two properties. The first is document, which refers to the document object present in the browser context. If the context is not running in Browser context then document is set to null. The other property is element, which stores an element that will be used by some instance during the execution of the collectable tree.

Kind: global class Summary: Create an object to give context to instance executions.

context.clone() ⇒ Context

Create a new instance of the same class and assign the values of the object from which it was created. The reason for this method is that the values of the element property can be updated or modified without affecting the original context object, since the new instance is a completely independent object from the original one. Something that does not happen with a simple copy of objects.

Kind: instance method of Context Summary: Clones a context instance. Returns: Context - Object that represents the context or the element that is being passed on nested instances executions. Example (Clone an existing Context instance.)

const context = new Context();
const newContext = context.clone();

context.update(element) ⇒ Context

Update the context object by adding a new value for the element property. First, clone the existing Context object using the clone () method and then update the value of element.

Kind: instance method of Context Summary: Update the Context object. Returns: Context - Object that represents the context or the element that is being passed on nested instances executions.

ParamTypeDescription

element

string | Array | object

Stores an element that will be used by some instance during the execution of the collectable tree.

Example (Update an existing Context instance.)

const context = new Context();
const newContext = context.update('new element');

context.getElement() ⇒ string | Array | Object

Gets the value of the element property. If element is equal to null then it will return the value of the document property.

Kind: instance method of Context Returns: string | Array | Object - an element that will be used by some instance during the execution of the collectable tree. Example (Getting the node to be used for nested executions)

const context = new Context();
...
const element = context.getElement();

SelectorError

Custom error to be used by the QuerySelector class in case an element or series of elements does not exist in the DOM.

Kind: global class

new SelectorError(selector)

ParamTypeDescription

selector

string

CustomError

Custom error to be used by the Collectable class to identify if the error was created by the error or require methods of the same class.

Kind: global class

new CustomError(message)

ParamTypeDescription

message

string

Custom message.

CollectableError

Custom error to be used by the Collectable class to return any error caused in the execution of the Collectable tree and that does not have a Collectable default method specified.

Kind: global class

new CollectableError(message, history)

ParamTypeDescription

message

string

Custom message

history

Array.<string>

An array of messages of previous executions in the collectable chain.

Logger

Log useful information and register errors.

Kind: global class

Logger.error(origin, message)

Error level.

Kind: static method of Logger

ParamTypeDescription

origin

string

Name of the class, function of where the log comes from.

message

string

Message.

Logger.warn(origin, elements, message)

Warn level.

Kind: static method of Logger

ParamTypeDescription

origin

string

Name of the class, function of where the log comes from.

elements

object

Any element to record its value, data type and instance.

message

string

Message

Logger.info(origin, elements, message)

Info level.

Kind: static method of Logger

ParamTypeDescription

origin

string

Name of the class, function of where the log comes from.

elements

object

Any element to record its value, data type and instance.

message

string

Message

Logger.debug(origin, elements, message)

Debug level.

Kind: static method of Logger

ParamTypeDescription

origin

string

Name of the class, function of where the log comes from.

elements

object

Any element to record its value, data type and instance.

message

string

Message

StrategyManager

Provides a common method to share across the different strategies managers.

Kind: global class

StrategyManager.lookUp(element, strategies) ⇒ Promise.<object>

Search or look up the best strategy.

Kind: static method of StrategyManager Returns: Promise.<object> - An object that represents a promise for a specific strategy.

ParamTypeDescription

element

Any

A criterion to be evaluated.

strategies

Set.<object>

Available strategies.

TypeValidator

Provides a static method that checks the data type of an incoming value.

Kind: global class

TypeValidator.check(value, [type]) ⇒ Error | void

Check the entry value if its data type of instance is the same than the type argument.

Kind: static method of TypeValidator Returns: Error | void - - Throws an error if the type does not match the value's data type.

ParamTypeDefaultDescription

value

any

Any value for which want to check the data type.

[type]

string | object

"string"

Data Type: string, number, array, object, boolean, function, CollectionCollector, Collector, NodeCollector.

Example (Checking if values is String)

TypeValidator.check('name');

Example (Checking if value is Number)

TypeValidator.check(5, 'number');

TypeValidator.deepCheck(value, type)

Check recursively if he entry value if its data type of instance is the same than the type argument.

Kind: static method of TypeValidator

ParamTypeDescription

value

any

Any value for which want to check the data type.

type

object | string

Data Type: string, number, array, object, boolean, function, CollectionCollector, Collector, NodeCollector. //TODO: Examples.

LazyLoadHandler

Loads all DOM elements that are handled by a LazyLoad.

Kind: global class

LazyLoadHandler.execute(buttonSelector, [stopLoad])

Executes the loading of all the elements by providing a clickable element selector, for example, a button.

Kind: static method of LazyLoadHandler

ParamTypeDefaultDescription

buttonSelector

string

CSS Selector that represents a button.

[stopLoad]

function

(button) => button && !(button?.disabled)

A function that returns a boolean to control when the event needs to be stopped. By default function is (button) => button && !(button?.disabled), where the LazyLoadHandler needs to stop when the button is disabled.

Pagination

Handles the pagination of an HTML section.

Kind: global class

Pagination.execute(buttonSelector, waitFor, [stopLoad])

Create a generator object with the new rendered document.

Kind: static method of Pagination

ParamTypeDefaultDescription

buttonSelector

string

CSS selector of the button that triggers the action to go to the next pagination.

waitFor

object

Selector of an element and timeout in milliseconds to wait for.

waitFor.timeout

number

The number of milliseconds to wait for.

waitFor.selector

string

A selector of an element to wait for.

waitFor.customFunction

function

A function that performs others waiting processes.

[stopLoad]

function

(button) => button

A function that returns a boolean to control when the event needs to be stopped. By default function is (button) => button, where the Pagination functionality needs to stop when the button is not available.

Logger.(elements) ⇒ Array.<object>

Extract useful information from elements and give them a specific format inside a list.

Kind: global function Returns: Array.<object> - - List of objects that represent an element.

ParamTypeDescription

elements

object

Any element to record its value, data type and instance.

clickAndWait(elementSelector, waitFor)

Click an element in the DOM and then wait for another event like a timeout, a selector being loading in the DOM or wait for the completion of an asyncronous function.

Kind: global function

ParamTypeDescription

elementSelector

string

CSS Selector to click on.

waitFor

object

Options to wait for.

waitFor.timeout

number

Timeout.

waitFor.selector

string

A CSS Selector.

waitFor.customFunction

function

A function that performs others waiting processes.

Classes

Collection

Execute a set of queries.

IterableAccessor

Iterates over each item in context created by a collection, returning it to be collected upon.

ProxyAccessor

Default ContextAcessor, used when there is not contextProcessor in Collector.

CollectionCollectorFactory

Create a Collector instance for collecting a collection for each iterable item in the context.

ElementCollectorFactory

Creates a Collector instance that returns a NodeList of DOM elements.

OptionCollectorFactory

Creates a Collector instance that returns a list of combinations generated by Options intances.

Option

Creates an iterable object from a series of options and at the same time, the iterable object has a function that is executed in each iterative cycle (next method call) to select a specific option value.

OptionStrategyManager

Manage the OptionStrategies.

Functions

OptionCollectorFactory.(options)object

Execute normalization actions for each of the options. For example, adding default values in queries.

OptionCollectorFactory.(collector, options)

Add queries to the Collector instance to extract each of the options in the Collector run.

Collection

Execute a set of queries.

Kind: global class

new Collection(queries)

ParamTypeDescription

queries

Object

Set of queries.

queries.*

string | function | Query

QUery.

collection.call(context) ⇒ Promise.<object>

Execute a set of queries.

Kind: instance method of Collection Returns: Promise.<object> - An object that represents a promise for a object with the results of the queries.

ParamTypeDescription

context

Context

Object that represents the context or the element that is being passed on nested queries or instances executions.

Example

        await page.evaluate( async () => {
            const data = new Collection({
                name: css('h1').property('innerText').single()
            });
                
            const context = new Context();
            console.log(await data.call(context)); // { name: 'Plato Plugin' }
        });

collection.postProcessor(customProcessor) ⇒ Collection

Add a new postProcessor to the list.

Kind: instance method of Collection Returns: Collection - Returns the current Collection instance.

ParamTypeDescription

customProcessor

function

A custom functionality that receives the query result and perform a process to return a transformed data.

IterableAccessor

Iterates over each item in context created by a collection, returning it to be collected upon.

Kind: global class

new IterableAccessor(collection)

ParamTypeDescription

collection

Collection instance that provides the new context.

iterableAccessor.call(context) ⇒ Promise.<Generator>

Create a generator to pass the new context.

Kind: instance method of IterableAccessor Returns: Promise.<Generator> - An object that represents a promise for a generator of context objects.

ParamTypeDescription

context

Context

Actual context object.

Example (Receives a Collection and returns an iterable of Context objects)

        await page.evaluate( async () => {
                        
            const data = new IterableAccessor(
                new Collection({
                    reviews: () => Array.from(document.querySelectorAll('#reviews > ul > li'))
                }).postProcessor(CollectionElementProcessor)
            );

            const context = new Context();

            for await(let newContext of data.call(context)) {
                console.log(newContext); // Returns the li elements inside of a Context object.
            }
        });

ProxyAccessor

Default ContextAcessor, used when there is not contextProcessor in Collector.

Kind: global class

proxyAccessor.call(context) ⇒ Generator.<Context>

Returns a generator with the incoming context.

Kind: instance method of ProxyAccessor Returns: Generator.<Context> - Returns the incoming context as part of a generator.

ParamTypeDescription

context

Context

Object that represents the context or the element that is being passed on nested queries or instances executions.

Example

        await page.evaluate( async () => {
                        
            const context = new Context().update('Custom Context');

            const data = new ProxyAccessor().call(context);

            let contextContainer = [];

            for await(let newContext of data) {
                contextContainer.push(newContext);
            }

            console.log(contextContainer[0].getElement()); // 'Custom Context'
            
        });

CollectionCollectorFactory

Create a Collector instance for collecting a collection for each iterable item in the context.

Kind: global class

new CollectionCollectorFactory(collector, queries)

Returns: Collector - A Collector instance.

ParamTypeDescription

collector

Collector

Collector instance.

queries

object

Set of queries.

ElementCollectorFactory

Creates a Collector instance that returns a NodeList of DOM elements.

Kind: global class

new ElementCollectorFactory(query)

Returns: Collector - A new instance of Collector.

ParamTypeDescription

query

function | Object | Query | string

A callabel object.

OptionCollectorFactory

Creates a Collector instance that returns a list of combinations generated by Options intances.

Kind: global class

new OptionCollectorFactory(options)

Returns: Collector - A Collector instance.

ParamTypeDescription

options

object

A series of options.

Option

Creates an iterable object from a series of options and at the same time, the iterable object has a function that is executed in each iterative cycle (next method call) to select a specific option value.

Kind: global class Summary: Creates an iterable object from options.

new Option(identifier, element, [strategy])

ParamTypeDescription

identifier

string

This tag identifies which option certain values ​​are linked to. For example, the CE, EE, and ECE values ​​are tied to an Edition option.

element

Element

DOM element.

[strategy]

object

Object that has functionalities to get options and set options. If no strategy is specified, then the Strategy Manager will assign one automatically.

[strategy.getOptions]

function

A function that creates an array of objects, in which each one of them contains two properties: value - which refers to the value property of an element that will be used to make the selection of the option. And, the second property is the dynamic value of the label parameter, which will contain the text of the value itself.

[strategy.setOption]

function

Function that performs a selection of an option based on a value that is specified as a parameter.

Example (Create an Option instance of a dropwdown (select element).)

const edition = new Option('edition', document.querySelector('#select_1573'));
const support = new Option('support', document.querySelector('#select_1583'));
const installation = new Option('installation', document.querySelector('#select_1593'));

Example (Create an Option instace of a group of buttons (div elements).)

// First, we need to create a function to get the options available based on the checkboxes
function getEditionAndSupportOptions(label, element) {

     return new Promise((resolve, reject) => {

             const options = element.querySelectorAll('div'); // Get all the div elements
             let result = [];
             
             for(let option of options) {
                 result.push({
                     value: option,
                     [label]: option.innerText
                 });
             }

             resolve(result);

             });
}

// We need to create the function to make the seleccion.

async function setOption(newElement) {
     newElement.click();
     return true;
}

// Now that we have out functions we can create our Option instance.

const edition = new Option('edition', document.querySelector('div.swatch-attribute.edition > div'), { getOptions: getEditionAndSupportOptions, setOptions });
 

option.call()

Creates an iterable object that contains the values of an HTML element and its respective selection function. The values collection and function execution is controlled by the JavaScript Iteration Protocols.

Kind: instance method of Option Example (Use the next generator method to get the values)

const edition = new Option('edition', document.querySelector('#select_1573'));

// Note: The use of next below refers to the method on Option instance.
const editionOptions = edition.next();

// Note: The use of next below refers to the iteration protocol method.
console.log(editionOptions.next()); // {value: { value: '22334', edition: 'CE' }, done: false };
console.log(editionOptions.next()); // {value: { value: '22335', edition: 'EE' }, done: false };
console.log(editionOptions.next()); // {value: { value: '22336', edition: 'ECE' }, done: false };
console.log(editionOptions.next()); // {value: undefined, done: true };

OptionStrategyManager

Manage the OptionStrategies.

Kind: global class

OptionStrategyManager.add(strategy)

Add or register a OptionStrategy sub-class.

Kind: static method of OptionStrategyManager

ParamTypeDescription

strategy

OptionStrategy

A specific implementation/sub-class of OptionStrategy.

Example

OptionStrategyManager.add(SelectStrategy);

OptionStrategyManager.lookUp(element) ⇒ Promise.<OptionStrategy>

Search or look up the best strategy.

Kind: static method of OptionStrategyManager Returns: Promise.<OptionStrategy> - Object that represents a promise to return a sub-class of OptionStrategy.

ParamTypeDescription

element

Element | Array.<Element>

DOM element that represents or has options.

Example (SelectStrategy matched)

        await page.evaluate(async () => {
            const selectElement = document.querySelector('#option-1');
            const strategy = await OptionStrategyManager.lookUp(selectElement);

            console.log(strategy === SelectStrategy); // true
        });

OptionCollectorFactory.(options) ⇒ object

Execute normalization actions for each of the options. For example, adding default values in queries.

Kind: global function Returns: object - The normalized options.

ParamTypeDescription

options

object

Set of options.

OptionCollectorFactory.(collector, options)

Add queries to the Collector instance to extract each of the options in the Collector run.

Kind: global function

ParamTypeDescription

collector

Collector

Collector instance.

options

object

Set of options.

Classes

CollectionElementProcessor

Creates a generator from a list of elements.

CollectionOptionProcessor

Creates a generator from a list of Option instances.

GroupStrategy

Knows how to get and set values from a group of options.

MissingStrategy

Default strategy.

SelectStrategy

Knows how to get and set values from SELECT DOM elements.

ToogleStrategy

Knows how to get and set values from an element with two states: TRUE/FALSE.

SelectorInterpreter

Creates a query chain from a custom selector.

InterpreterStrategyManager

Manage which interpretation strategy will process the custom selector.

All

Contains the specific logic to process nested structures. In case no parameter is passed to its constructor then it will return the previous result in the execution of the chain.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

CollectorSelector

Execute a Collector instance.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Css

Contains specific logic to extract elements from the DOM using a CSS selector.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Default

Returns a default value if there the expected result is not valid.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

SelectorDirectory

Provide a directory to store proxy function that will be used to instantiate the Implementation sub-classes. The directory is made up of the name of each proxy function and its sub-class constructor equivalent.

Elements

Creates a collector that will return a generator of a list of elements.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Init

Perform actions during initialization.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Iterate

Iterate a Collector instance against a set of queries.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Merge

Contains the specific Selector that allows concatenating other query Selectors.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Options

Creates a collector that will return a generator of a list of option values.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Post

Perform actions after data extraction.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Pre

Perform actions before data extraction.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Property

Contains the specific Selector to obtain properties of DOM elements.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Require

Returns an error if the result is not a valid value.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Select

Interprets Select Strings.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Single

Contains the specific Selector to check if a single element is expected as it turns out, otherwise throw an error.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

XpathSelector

Contains specific logic to extract elements from the DOM using a xpath selector.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Functions

CollectionElementProcessor.(element)boolean

Check the object is a generator.

CollectionElementProcessor.(elements)

Create a generator from a list of elements.

CollectionOptionProcessor.(options)Promise.<Generator>

It takes Option instances and calculates all possible values ​​of the combinations of those options.

CollectionElementProcessor

Creates a generator from a list of elements.

Kind: global class

CollectionElementProcessor.call(elementCollection) ⇒ Generator.<(Element|any)>

Execute the processor.

Kind: static method of CollectionElementProcessor Returns: Generator.<(Element|any)> - A generator with DOM elements.

ParamTypeDescription

elementCollection

object

A serie of DOM elements or generator.

Example (Receives a collection with a list of DOM elements an returns an array with elements)

const result = await page.evaluate(() => {
     const elements = { reviews: Array.from(document.querySelectorAll('#reviews > ul > li')) };
     const data = CollectionElementProcessor.call(elements);
     console.log(data); // [li, li]
});

CollectionOptionProcessor

Creates a generator from a list of Option instances.

Kind: global class

CollectionOptionProcessor.call(elementCollection) ⇒ Promise.<Generator>

Returns all possible values ​​of the options that were passed to Collection.

Kind: static method of CollectionOptionProcessor Returns: Promise.<Generator> - An object that represents a promise object for a generator containing all the possible values processed of each option.

ParamTypeDescription

elementCollection

object

A serie of elements or Options instances.

Example (Processing a series of options)

        const result = await page.evaluate(async () => {
                
            const options = {
                edition: document.querySelector('select#option-1'),
                support: document.querySelector('select#option-2')
            };

            const values = await CollectionOptionProcessor.call(options);

            let result = [];

            for await(let value of values) {
                console.log(value); // First Iteration: [{ value: '40', support: 'val-40' }, { value: '10', edition: 'val-10' }]
            }

        });

GroupStrategy

Knows how to get and set values from a group of options.

Kind: global class

GroupStrategy.getOptions(identifier, element) ⇒ Promise.<Array>

Extract a value that will be used to identify each option.

Kind: static method of GroupStrategy Returns: Promise.<Array> - Object that represents a promise for a list of objects.

ParamTypeDescription

identifier

string

Identifies the option values ​​referenced by the element in the DOM.

element

Element

DOM element that contains one or more values.

Example (Get options from a select element)

        await page.evaluate(async () => {
            const element = document.querySelectorAll('#div-1 > div');
            const result = await GroupStrategy.getOptions('edition', element);
            console.log(result); // [ { value: div, edition: 'div-val-10'}, { value: div, edition: 'div-val-20'}, { value: div, edition: 'div-val-30' }]
        });

GroupStrategy.setOption(element, value) ⇒ Promise.<Boolean>

Select a specific option.

Kind: static method of GroupStrategy Returns: Promise.<Boolean> - Object that represents a promise for the completion of a select action.

ParamTypeDescription

element

Element

DOM element that contains one or more values.

value

string | object

Value that will be used to select a specific option.

Example (Set second option to a div element)

        await page.evaluate(async () => {
            const parentElement = document.querySelector('#div-1');
            const element = document.querySelector('#div-1 > div:nth-child(2)');

            await GroupStrategy.setOption(parentElement, element);

            console.log(parentElement.getAttribute('value')); // 20
        });

GroupStrategy.match(element) ⇒ Promise.<Boolean>

Check if the particular strategy is suitable for a certain element of the DOM.

Kind: static method of GroupStrategy Returns: Promise.<Boolean> - Object that represents a promise for a TRUE or FALSE value.

ParamTypeDescription

element

Element

DOM element that contains one or more values.

Example (Match elements)

        await page.evaluate(async () => {
            const element = document.querySelectorAll('#div-1 > div');
            console.log(await GroupStrategy.match(element)); // true
        });

MissingStrategy

Default strategy.

Kind: global class

SelectStrategy

Knows how to get and set values from SELECT DOM elements.

Kind: global class

SelectStrategy.getOptions(identifier, element) ⇒ Promise.<Array>

Extract a value that will be used to identify each option.

Kind: static method of SelectStrategy Returns: Promise.<Array> - Object that represents a promise for a list of objects.

ParamTypeDescription

identifier

string

Identifies the option values ​​referenced by the element in the DOM.

element

Element

DOM element that contains one or more values.

Example (Get options from a select element)

        await page.evaluate(async () => {
            const selectElement = document.querySelector('#option-1');
            console.log(await SelectStrategy.getOptions('edition', selectElement)); // [{ value: '10', edition: 'val-10' }, { value: '20', edition: 'val-20' }, { value: '30', edition: 'val-30' }]
        });

SelectStrategy.setOption(element, value) ⇒ Promise.<void>

Select a specific option.

Kind: static method of SelectStrategy Returns: Promise.<void> - Object that represents a promise for the completion of a select action.

ParamTypeDescription

element

Element

DOM element that contains one or more values.

value

string | object

Value that will be used to select a specific option.

Example (Set second option to a select element)

        await page.evaluate(async () => {
            const selectElement = document.querySelector('#option-1');
            await SelectStrategy.setOption(selectElement, '20');
            console.log(selectElement.value); // '20'
        });

SelectStrategy.match(element) ⇒ Promise.<(Boolean|Error)>

Check if the particular strategy is suitable for a certain element of the DOM.

Kind: static method of SelectStrategy Returns: Promise.<(Boolean|Error)> - Object that represents a promise for a TRUE or FALSE value.

ParamTypeDescription

element

Element

DOM element that contains one or more values.

Example (Match a SELECT element)

        await page.evaluate(async () => {
            const selectElement = document.querySelector('#option-1');
            console.log(await SelectStrategy.match(selectElement)); // true
        });

ToogleStrategy

Knows how to get and set values from an element with two states: TRUE/FALSE.

Kind: global class

ToogleStrategy.getOptions(identifier, element) ⇒ Promise.<Array>

Extracts a value that will be used to identify each option.

Kind: static method of ToogleStrategy Returns: Promise.<Array> - Object that represents a promise for a list of objects.

ParamTypeDescription

identifier

string

Identifies the option values ​​referenced by the element in the DOM.

element

Element

DOM element that contains one or more values.

Example (Get options from a checkbox element)

        await page.evaluate(async () => {
            const selectElement = document.querySelector('#toogle');
            console.log(await ToogleStrategy.getOptions('installation', selectElement));
            // [{ value: { click: {}, selection: true }, installation: true }, { value: { click: {}, selection: false }, installation: false }]
        });

ToogleStrategy.setOption(element, value) ⇒ Promise.<void>

Select a specific option.

Kind: static method of ToogleStrategy Returns: Promise.<void> - Object that represents a promise for the completion of a select action.

ParamTypeDescription

element

Element

DOM element that contains one or more values.

value

string | object

Value that will be used to select a specific option.

Example (Select an input element)

        await page.evaluate(async () => {
            const inputElement = document.querySelector('#toogle > input');
            await ToogleStrategy.setOption(inputElement, { click: inputElement, selection: true });
            console.log(inputElement.checked); // true

ToogleStrategy.match(element) ⇒ Promise.<(Boolean|Error)>

Check if the particular strategy is suitable for a certain element of the DOM.

Kind: static method of ToogleStrategy Returns: Promise.<(Boolean|Error)> - Object that represents a promise for a TRUE or FALSE value.

ParamTypeDescription

element

Element

DOM element that contains one or more values.

Example (Match an element)

        await page.evaluate(async () => {
            const selectElement = document.querySelector('#toogle');
            console.log(await ToogleStrategy.match(selectElement)); // true
        });

SelectorInterpreter

Creates a query chain from a custom selector.

Kind: global class

new SelectorInterpreter(selector)

Returns: Query - A query instance.

ParamTypeDescription

selector

string

A custom selector.

Example

    SelectorDirectory.register(SelectorInterpreter);
    await page.evaluate(async () => { 
    
        const data = ( function () {

            const select = SelectorDirectory.get('selectorinterpreter');

            return new Collection({
                name: select('h1')
            });
            
        } )();
        
        const context = new Context();
        console.log(await data.call(context)); // Plato Plugin
    });

InterpreterStrategyManager

Manage which interpretation strategy will process the custom selector.

Kind: global class

InterpreterStrategyManager.add(strategy)

Add a new strategy.

Kind: static method of InterpreterStrategyManager

ParamTypeDescription

strategy

object

A strategy that can process a custom selector.

Example

        await page.evaluate(() => {
            InterpreterStrategyManager.add(InterpreterElementStrategy);
            InterpreterStrategyManager.add(InterpreterInnerTextStrategy);
            InterpreterStrategyManager.add(InterpreterPropertyStrategy);
        });

InterpreterStrategyManager.lookUp(selector) ⇒ object

Search among the available strategies which of them is the most suitable to process a specific selector.

Kind: static method of InterpreterStrategyManager Returns: object - A strategy that can process the custom selector.

ParamTypeDescription

selector

string

Custom selector.

Example ({h1} gets InterpreterElementStrategy)

        await page.evaluate(async () => { 
            const interpreter = InterpreterStrategyManager.lookUp('{h1}');
            console.log(interpreter === InterpreterElementStrategy ? true : false); // true
        });

All

Contains the specific logic to process nested structures. In case no parameter is passed to its constructor then it will return the previous result in the execution of the chain.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Kind: global class Summary: Process nested structures.

CollectorSelector

Execute a Collector instance.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Kind: global class

Css

Contains specific logic to extract elements from the DOM using a CSS selector.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Kind: global class Summary: Extracts Dom Elements.

Default

Returns a default value if there the expected result is not valid.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Kind: global class

SelectorDirectory

Provide a directory to store proxy function that will be used to instantiate the Implementation sub-classes. The directory is made up of the name of each proxy function and its sub-class constructor equivalent.

Kind: global class Summary: Keep a record of Selector subclasses.

SelectorDirectory.register(selector)

Saves a record of the subclass within the registry as a function.

Kind: static method of SelectorDirectory

ParamTypeDescription

selector

Object

Constructor class such as Css, Property, Single, etc.

Example (Use in Selector class)

SelectorDirectory.register(this); // 'this' referring to the class constructor that is extending Implementation.

SelectorDirectory.get(name) ⇒ function

Returns the proxy function that was previously registered within the registry object.

Kind: static method of SelectorDirectory Returns: function - Proxy function that will create the instance of the sub-class.

ParamTypeDescription

name

String

Name of the function stored in the registry object. It will be the lowercase version of the sub-class name.

Example (Use in Selector class)

// Create the intance of the functionality
const queryImplementation = SelectorDirectory.get(prop);
const queryInstance = new queryImplementation(definition, this.query);

SelectorDirectory.iterate()

Creates an iterable object of all registries inside the registry object.

Kind: static method of SelectorDirectory

Elements

Creates a collector that will return a generator of a list of elements.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Kind: global class

Init

Perform actions during initialization.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Kind: global class

Init.register()

Modifies the registry process in SelectorDirectory class.

Kind: static method of Init

Iterate

Iterate a Collector instance against a set of queries.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Kind: global class

Merge

Contains the specific Selector that allows concatenating other query Selectors.

The class itself is not open to developer use, but rather is used by a proxy function to build the instance. See the example for more information.

Kind: global class Summary: Concatenates query Selectors