Skip to main content

MongoDB API Tutorial

Prerequisites

This tutorial expects familiarity with MongoDB API querying fundamentals. Qarbine interacts with the MongoDB API in a native manner and fully supports the document data model. This enables direct access to the MongoDB database features ranging from the aggregation pipeline, to full text search (FTS), to AI driven vector search ($vectorSearch). Interaction with the database is from a connection string which enables you the flexibility to access data on real-time, secondary, or analytic nodes. Free MongoDB Atlas DBaaS instances can be deployed by visiting Getting Started with MongoDB Atlas at https://www.mongodb.com/lp/cloud/atlas/try4. There are also many querying tutorials available on the web and numerous learning resources at MongoDB University (https://learn.mongodb.com/).

Overview

This tutorial covers aspects of Qarbine and fundamental use of the general MongoDB querying language and answer set handling. MongoDB databases store data as JSON/BSON documents. Each document represents a record and can have varying fields. Documents can be nested to express hierarchical relationships and store structures like arrays. Querying from Qarbine uses the MongoDB Querying Language (MQL) and can use basic find() interactions or the more powerful aggregation pipeline interface. Qarbine passes the query along to the backend MongoDB connection string identified endpoint and accepts the answer set list of documents.

Further query interaction information can be found by navigating to the Tools tab

  

Then, selecting the categories shown below.

  

Defining a Data Source

Overview

A Data Source is a Qarbine component responsible for retrieving data. At a high level it has a name, a description and some arbitrary query string which when sent to the associated Qarbine Data Service endpoint returns some data. The overall execution flow for an analysis, including the optional prompt component, is shown below.

  

A single data source can be referenced by name from multiple Qarbine template components. This enables a single point of change when perhaps, an index is added, or some other query tweak is necessary. The alternative is to attempt to find all templates impacted by a schema or index change for example. The component reusability is especially beneficial when team members have varying roles and skills.

Opening the Tool

You can access the Data Source Designer in several ways. Tools can be opened from the signOn page, the home page, and from various tools using the hamburger    menu.

From the Home tool.

  

Alternatively you can open the tool from the hamburger    menu on each tool.

  

. . .

  

Pressing the control key during the menu click opens the tool in another tab.

Note that pressing the control key while clicking this option opens the tool in another tab. This is a convenient way to have multiple Qarbine tabs open and quickly navigate between them.

From the sign on page

  

The core layout of the Data Source Designer is depicted below.
  

Query Language Options

The Qarbine driver for the MongoDB API permits either MongoDB Query Language or SQL queries. The latter is only supported when the endpoint is configured for MongoDB Atlas SQL. Qarbine previews the query to determine which underlying endpoint service query dialect to call.

Example

In the tool the first thing to do is specify the data context. This is done by selecting the Qarbine data service in the drop down and then the database from the second drop down. The “Sample Data Service” is one defined as part of the standard installation. Qarbine administrator may choose to disable it as they please.

  

The top area is for entering the query criteria. The data source below retrieves a list of portfolios from the portfolio collection.

  

A sample result element is shown below. Notice the with embedded documents and arrays.

  

To see the general structure of a collection simply choose it from the left-hand drop down member as shown below. This is an approximate schema because of sampling and the flexibility of the MongoDB document model.

  

A data source component can be saved and reused by any number of templates. Other team members using the component do not need to become experts in the MongoDB Query Language or the underlying data schema.

Data sources can be made more flexible by using variables within their query. A Qarbine prompt component can be defined to prompt the user for the value. A new data source can be defined with the query

db.portfolio.find( {isIRA: @isIRABoolean} )

Since there is a variable reference (denoted by the ‘@’ prefix in ‘@isIRABoolean’) running this within the Data Source Designer presents a generic prompt as shown below.

  

The user can then set the data type to boolean and click the checkbox.

  

Click the OK button to continue. A sample result is shown below.

  

The component can be saved with the name “All portfolios for @isIRA”.

Prompt Integration

Overview

Qarbine prompts provide a way to obtain runtime values and variables for data source and template execution. To avoid hardcoding, prompts can use macro formulas to run queries which populate list widgets. Prompts are defined in a no code manner using the Prompt Designer. See that document for details on the powerful, no-code features of the Prompt Designer. Shown below is the execution flow when there is a Prompt component.

  

Opening the Tool

You can access the Prompt Designer in several ways. Tools can be opened from the signOn page, the home page, and from various tools using the hamburger    menu.

From the Home tool.

  

Alternatively you can open the tool from the hamburger    menu on each tool.

  

  

Pressing the control key during the menu click opens the tool in another tab.

Shown below is an example of a basic prompt.

  

Access the Prompt Designer. The prompt starts with no elements but will eventually have the following elements.

  

In the left hand area either double click    or drag and drop either into the elements table area. The first is a heading as defined below.

  

When changes are made to the prompt element the top area icon will be changed from

  

to

  

Update the element as shown above.
You have to exit out of any entry field for the change to be noticed. Click    to save the element and update the sample overall prompt to the right.
In the left hand area widgets either double click   or drag and drop either into the elements table area. Select the element. The second element uses a checkbox as defined below.

  

Update the element as shown above.

To test the prompt click

  

Interact with the dialog as desired.

  

Click OK to close the dialog.

The Prompt Designer changes tabs and shows the following

  

To save the prompt click

  

In the catalog prompt navigate to the target folder.
Name the prompt “Prompt for @isIRABoolean”.
Click the OK button.

Data sources, templates and other components can reference prompts to obtain runtime values. Much more complex prompts are possible. See the Prompt Designer Tool guide for details.

A data source or template can reference this prompt to obtain the isIRABoolean runtime variable value. For now, we’ll associate the data source with it. This is done within the Data Source Designer. Navigate to the Data Source Designer tool using one of the tool navigation options described above.

To access the data source’s properties dialog click

  

Within the properties dialog navigate to the Prompt tab as shown below.

  

Click the option highlighted below

  

Click the recents button highlighted below.

  

Choose the just created prompt.

  

To close the Recents dialog click

  

The reference is filled into the Data Source Properties dialog.

  

Close the properties dialog by clicking

  

To save the Data Source in the catalog click

  

Running the data source now presents the prompt to obtain the variable value. Try it by clicking

  

The prompt dialog is presented.

  

Make your selections and click the OK button. The runtime value propagates along the execution path, is placed into the query variable, and the query is run. The results are then shown in the tool.

Template Designer Use

General Interactions

To specify your templates Qarbine presents a grid of cells across lines belonging to various template sections. Similar to Excel, Qarbine formulas are defined by first clicking on the grid area and then typing into the formula entry field. The right hand side of the Template Designer shows the properties of the cell. There are ‘regular’ cells and ‘custom’ cells. The latter are for elements such as images and interactive buttons. Qarbine provides a large variety of custom cells for flexible presentation and interaction options.

Layout Grid Paradigm

The Qarbine layout paradigm is similar to an outline with various levels. The template has a main data retrieval and one or more ‘body’ sections. The simplest layout would have a report header, a 1.1 group header, a 1.1.1 body, a 1.1. Group summary and a report summary sections. Sections can have multiple lines and, like cells, there are many options for controlling their processing and presentation. Below is a snapshot of the basic outline structure.

  

Data Handling

In the “hello world” case the main retrieval’s data elements are formatted within the 1.1.1 body.
When the data to be processed has multiple lists (AKA embedded arrays) then multiple groups need to be defined. Groups provide the mechanism for iterating through lists. Groups can contain inner groups. In the stock portfolio example there is a main retrieval for the list of portfolios and each portfolio has a list of stocks. In this case the 1.1 group header would show general portfolio information and there would be a data retrieval defined for 1.1 group for the list of stocks. The stock details would be formatted within the 1.1.1 body.

Many modern data structures also have multiple lists. In the sample portfolio data the list of stocks and the lists of bonds would be an example. Qarbine’s flexible analysis engine supports multiple peer groups. Groups are added by selecting any line in the group in the left hand side and right clicking for the popup menu.

  

The group options are toward the bottom of the popup menu.

  

Embedded document data is typically accessed using dot path notation. For example,

#profile.phone

Formulas

Cell formulas can reference field names using the ‘#’ prefix and formula results can be saved into variables. Here is a simple example,

set value  = #price * #quantity

Excel generally uses arbitrary cell references like “F5 * F6” in cell formulas. Qarbine promotes the use of meaningful field names and variables. As a result, formulas are much easier to understand. To reference a variable in another formula use the ‘@’ prefix. For example,

set totalCost = @value * @taxRate

Formulas can be used in numerous areas including processing conditions, background colors, foreground colors and pagination rules. There are over 400 macro functions available in Qarbine. Simply click the ‘...’ button or control-E to access the formula builder dialog.

Defining an Analysis Template

Opening the Tool

You can access the Template Designer in several ways. Tools can be opened from the signOn page, the home page, and from various tools using the hamburger    menu.

From the Home tool.

  

Alternatively you can open the tool from the hamburger    menu on each tool.

  

. . .

  

Pressing the control key during the menu click opens the tool in another tab.

Overview

A template defines how to process the data being retrieved from Data Source queries and other data expressions. It also defines formulas, formatting options, and other analysis and presentation options. The overall execution flow for an analysis, including the optional prompt component, is shown below

  

Using the Template Designer

A template can be created to consume the deeply nested portfolio objects from the data source. Running the template about to be described presents the prompt.

  

Sample results are shown below.

  

The template’s main properties of interest are shown below.

  

. . .

  

. . .

  

A template can reference a prompt content in a similar manner as a data source. The data source referenced above already references the prompt component.

Data retrieval approach can be seen on the right hand side by clicking an area without a cell and choosing the option noted. Groups 1.1.1 and 1.1.2 each extract embedded arrays by name from the main retrieval’s portfolio variable. They also set element variables for use in formulas.

  

The 1.1 group has 2 inner groups. One group iterates through the stocks and the other iterates through the bonds. The cell layout is depicted below.

  

The 1.1 group header cells are.

  

The 1.1.1 group iterates through the portfolio’s stocks. The group’s properties have

  

. . .

  

Here we chose to set the variable named stocksElement to the active element of the stocks list as it is iterated through. The ‘@current’ variable is always set. Below we reference the various variable fields using dot path notation.

The 1.1.1.1 body cells are.

  

The last cell uses the ‘#’ field shortcut notation.

The 1.1.1 group summary cell is

  

The 1.1.2 group iterates through the portfolio’s bonds. The group’s properties have

  

. . .

  

The pattern is similar to the one in the stock’s group. The 1.1.2.1 body cells are.

  

The 1.1.2 group summary cell is

  

Notice the use of the “sum()” macro to accumulate the bond values.

The 1.1 group summary contains

  

Here we reference the various variables to compute the overall portfolio value.

Below is an example of output from a more elaborate template using custom cells and other formatting features.

  

To save the template click

  

In the catalog prompt navigate to the target folder.

Name the template “Portfolios for @isIRABoolean”.

Click the OK button.

The template is now saved in the shared catalog.