Couchbase
Summary
This document covers the information to gather from Couchbase in order to configure a Qarbine data service. The data service will use the Qarbine Couchbase driver. You can define multiple data services that access the same Couchbase endpoint though with varying credentials. Once a data service is defined, you can manage which Qarbine principals have access to it and its associated Couchbase data. A Qarbine administrator has visibility to all data services.
Overview
Couchbase is a NoSQL database which provides an SQL-like querying interaction. For details, see the Couchbase home page at https://couchbase.com.
Enabling Qarbine access to your cluster data involves the following steps:
- Determining the Couchbase URL and bucket,
- Adding or identifying a Couchbase user account and its password,
- Enabled IP access and
- Defining a Qarbine Data Service.
Couchbase Information
Endpoint
Navigate to your Couchbase console. For Couchbase Capella it is at
https://cloud.couchbase.com/
Information on adding users can be found at
https://docs.couchbase.com/server/current/manage/manage-security/manage-users-and-roles.html
To locate your Couchbase URL navigate to the databases page by clicking
Click on the cluster of interest.
Click
Copy the public connection string
This value is used as the “server template” in the Qarbine data service.
Another DNS name of interest is the hostname shown below.
Database Account
You should have identified a user account and its password for Qarbine readonly interactions. For reference shown below would be an example of creating a user with just these permissions. You may wish to limit bucket access as well based on your requirements.
For more details see https://docs.couchbase.com/cloud/clusters/manage-database-users.html.
A starting page on the Capella console for managing cluster access is at https://cloud.couchbase.com/database/settings/access-control
Below is an example of a cluster access account.
The bucket level access should be done for each of the buckets for the given user.
More information is below. Once satisfied click
Buckets and Scopes
Conceptually,
- Buckets map to SQL databases,
- Scopes map to SQL schemas, and
- Collections map to SQL tables.
To see your buckets navigate to the Couchbase cluster tab shown below.
Couchbase Sample Buckets
Couchbase provides several data sets to quickly start interacting with Capella. The Qarbine samples in the “example/Couchbase” folder reference the data services named “Couchbase- XXX”. This Couchbase page provides some background on the sample data.
https://docs.couchbase.com/server/current/manage/manage-settings/install-sample-buckets.html
To load it, log in to Capella and navigate to the cluster of interest.
Click
For each data set of interest select it and then click.
Below is an example snapshot after some of the data sets are loaded.
The left hand side panel displays some of the examples loaded.
Expanding the first bucket shows
Expanding the second bucket shows
Expanding the third bucket shows
Identify the scope of interest for each bucket. Usually there is just one, well-named scope that is the one to use for Qarbine querying. The Qarbine database value described below is formatted with the bucket name, a space, and the scope name.
Bucket | Scope of Interest | Qarbine “Database” value |
---|---|---|
beer-sample | _default | beer-sample _default |
color-vector-sample | color | color-vector-sample color |
travel-bucket | inventory | travel-bucket inventory |
The Qarbine catalog has examples for the 3 buckets above. There are also 3 corresponding data services defined, one for each of the buckets.
NOTE - Each of these 3 Couchbase data services must be configured to reference your Couchbase cluster containing the sample bucket data as described below. They must also be enabled by choosing the option shown below.
When the Qarbine data service’s database field is empty, you will have to fully qualify the collection path in your queries with Bucket.Scope.Collection. If there are dashes in any of the pieces then use tick marks to enclose the string. For example, `travel-sample`.inventory.hotel.
If a user tries to run an example and that data service references data it doesn't have permission for then an error will be returned. An example is shown below.
Allowing an IP address
Add the Qarbine host IP address to the cluster’s list of allowed IPs using the information at
Access this area by clicking the highlighted sidebar item shown below.
Qarbine Data Service Configuration
Overview
The Qarbine data service requires
- Qarbine compute node URL,
- Couchbase URL,
- Couchbase user and password, and
- Couchbase default bucket.
Compute Node Preparation
Determine which compute node service endpoint you want to run this data access from. That URL will go into the Data Service’s Compute URL field. Its form is “https://domain:port/dispatch”. A sample is shown below.
The port number corresponds to a named service endpoint configured on the given target host. For example, the primary compute node usually is set to have a ‘main’ service. That service’s configuration is defined in the ˜./qarbine.service/config/service.main.json file. Inside that file the following driver entry is required
"drivers" :[
. . .
"./driver/couchbaseDriver.js"
]
The relevant configuration file name for non primary (main) Qarbine compute nodes is service.NAME.json. Remember to have well formed JSON syntax or a startup error is likely to occur. If you end up adding that entry then restart the service via the general command line syntax
pm2 restart <service>
For example,
pm2 restart main
or simply
pm2 restart all
Defining the Qarbine Data Service
Within the Qarbine Administration tool navigate to Data Services
Click
Enter the name, description, and Qarbine compute node endpoint URL.
Choose
For the server template enter
couchbases://THE_COPIED_URL_FROM_ABOVE
For the server options enter
userName="THE_USER",
password="THE_PASSWORD",
configProfile = "wanDevelopment"
Optionally you may specify options such as:
- consoleLogLevel
- ssl (the default is “no_verify”)
- tls_verify=none
- kvTimeout
- kvDurableTimeout
- viewTimeout
- queryTimeout
- analyticsTimeout
- searchTimeout
- managementTimeout
Information about server options is available by clicking the icon shown below.
You can reference environment variables using the syntax %NAME%. Any strings should be quoted and the key\value pairs separated by commas. For further information see
https://docs.couchbase.com/go-sdk/current/howtos/troubleshooting-cloud-connections.html
https://docs.couchbase.com/nodejs-sdk/current/ref/client-settings.html
The “database” field here maps to a Couchbase bucket and scope name with a space separator as described above.
When the Qarbine data service’s database field is empty, the query author will have to fully qualify the collection path in your queries with Bucket.Scope.Collection. If there are dashes in any of the pieces then tick marks are used to enclose the string. For example, `travel-sample`.inventory.hotel.
After you have entered the above information save the data service by clicking .
Next, test the settings by clicking on the icon noted below.
You should see the following general dialog
The data service will be known at the next log on time.
Data Service Query Interactions
For retrieving data from Couchbase Qarbine supports using a JSON specification. The format is
{
useSearch: true | false,
bucket: aString,
scope: aString,
index: aString, ← required.
other fields per Couchbase documentation
}
The useSearch flag determines whether the search or query endpoint is used. The configured Qarbine Data Service’s server template prefix (couchbase:// vs couchbases://) determines whether the HTTPS secure endpoint is used. The cbHostName value is the text after couchbase(s):// in the server template. A sample secure server template is
couchbases://cb.abcdefghijk.cloud.couchbase.com
Its cbHostName is
cb.abcdefghijk.cloud.couchbase.com
Protocol | Type | Endpoint |
---|---|---|
HTTP | query | http://${cbHostName}:8093 |
search | http://${cbHostName}:8094 | |
HTTPS | query | https://${cbHostName}:18093 |
search | https://${cbHostName}:18094 |