JSON messages are commonly used to transfer data between an App Service and an App Client. When the API response contains a large amount of data, it may be necessary to break the response into smaller pieces. Pagination is a common technique used to divide the data into multiple messages (pages) to make it easier to handle and send it via a WebSocket connection.
In this example, we will implement an API that returns a list of items (in our case, letters of the phonetic alphabet) from the database. The requested items are delivered in multiple messages, so it is possible to deliver any number of items using this approach.
In our example we will use the following database table:
id | letter
----+----------
1 | Alpha
2 | Bravo
3 | Charlie
. | ...
24 | Xray
25 | Yankee
26 | Zulu
Initial request from our client to the App Service is done with following JSON message, limiting the number of items returned in one message by 3:
{"api":"Alphabet","mt":"GetLetters","limit":3}
The App Service delivers following response to the initial message:
{"api":"Alphabet","mt":"GetLettersResult","letters":["Alpha","Bravo","Charlie"],"more":3}
To ask for more entries to be delivered, following message is sent to the App Service:
{"api":"Alphabet","mt":"GetLetters", "limit":3, "more":3}The last response will not have the "more" property set, indicating that there are no more entries to request:
{"api":"Alphabet","mt":"GetLettersResult"}To start requests from App UI in myApps client we will use the sendSrcMore function of the AppWebsocket Connection Library. The difference of the sendSrcMore function to the send function is an integrated auto request function, that repeat the initial request, until all items are delivered by the App Service. On every response from App Service the sendSrcMore function checks if the property "more" is set and if so, the "more" property and its value is attached to the initial request and sent again to the App Service.
In our example, we send the initial request and provide a callback function, that handles responses and logs the letters to the console (innovaphone-cppmoreexample.js):
// create UI container before AppWebsocket connection is created
var lettersContainer = this.add(new innovaphone.ui1.Div("display: flex; flex-wrap: wrap;", ""));
var app = new innovaphone.appwebsocket.Connection(start.url, start.name);
app.checkBuild = true;
app.onconnected = app_connected;
function app_connected(domain, user, dn, appdomain) {
// send the initial request
app.sendSrcMore({ api: "Alphabet", mt: "GetLetters", limit: 10 }, callback);
}
function callback(msg) {
// process response, add letters to UI container
if ("letters" in msg && Array.isArray(msg.letters)) msg.letters.forEach(function (letter) {
lettersContainer.add(new innovaphone.ui1.Div("font-size:30px; margin: 10px; text-align:center", letter));
});
}
On the App Service, implemented in JavaScript, the libraries JsonApi and Database are used.
In the case of an App Service implemented in C++, the following libraries are used: C++ JSON Library, C++ Framework for JSON APIs, C++ Database Library and C++ Tasks Library.
The preparation of the database for our example is done in two steps: creating a table "alphabet" and populating it with records.
The config.json file is used to create the required tables for the JavaScript App Service:
{
"$schema": "runtime-config.schema.json",
"javascript": {
"eval": [
"innovaphone-jsmoreexampleservice.js"
]
},
"database": {
"init": [
{
"cmd": "table",
"name": "alphabet",
"init": [
{
"cmd": "column",
"name": "id",
"type": "BIGSERIAL PRIMARY KEY NOT NULL"
},
{
"cmd": "column",
"name": "letter",
"type": "VARCHAR(32) UNIQUE NOT NULL"
}
]
}
]
}
}For the C++ App Service we will use the constructor of TaskDbInit class (see later code example).
To populate the table 'alphabet' with example data, following SQL statement can be used:
INSERT INTO alphabet (letter) VALUES ('Alpha'), ('Bravo'), ('Charlie'),
('Delta'), ('Echo'), ('Foxtrot'), ('Golf'), ('Hotel'),
('India'), ('Juliet'), ('Kilo'), ('Lima'), ('Mike'),
('November'), ('Oscar'), ('Papa'), ('Quebec'), ('Romeo'),
('Sierra'), ('Tango'), ('Uniform'), ('Victor'), ('Whiskey'),
('Xray'), ('Yankee'), ('Zulu');To run this statement, connect to the AP via SSH (default user admin, password ipapps). Switch to root "su -" followed by root password (default iplinux), and start Postresql console "psql -d databasename".
In general, we need to implement the following functionality in the App Service:
An App Service can be implemented in JavaScript or C++, so both variants are described separately. In both cases, a standard Visual Studio template project is used as the starting point.
In the following implementation of a JavaScript App Service, we collect the input parameters from the "limit" and "more" properties, and set default values, if none are provided. The next step is to prepare a response object and execute an SQL query. When the database response is ready, we add result elements to the response object and send it as an encoded JSON message.
innovaphone-jsmoreexampleservice.js:
new JsonApi("Alphabet").onconnected(function(conn) {
conn.onmessage(function (msg) {
var obj = JSON.parse(msg);
if (obj.mt === "GetLetters") {
// if no "limit" property provided or greater than 10, set it to 10
var limit = "limit" in obj && obj.limit <= 10 && obj.limit > 0 ? obj.limit : 5;
// if no "more" property provided (in case of initial request), set offset to 0
var offset = "more" in obj ? obj.more : 0;
var response = { api: "Alphabet", mt: "GetLettersResult", src: obj.src };
Database.exec("SELECT * FROM alphabet WHERE id>" + offset +" ORDER BY id ASC LIMIT " + limit)
.oncomplete(function (data) {
var last = undefined;
var letters = [];
data.forEach(function (row) {
letters.push(row.letter);
last = row.id; // last id is used as value of returned "more" property
});
if (letters.length > 0) response.letters = letters;
if (last !== undefined) response.more = last;
conn.send(JSON.stringify(response));
})
.onerror(function (error, errorText, dbErrorCode) {
response.error = true;
conn.send(JSON.stringify(response));
});
}
});
});The implementation is based on the assumption we have created a new Visual Studio project with the name "cppmoreexample". The complete project cppmoreexample is also available on GitHub.
Following classes are added to the template C++ App Service implementation, after a default project is created:
In the AppInstance class constructor implementation, after member initialisation, we add a new the JsonApiContext for our API (cppmoreexample.cpp):
RegisterJsonApi(this);
this->alphabetApiContext = new AlphabetApiContext("Alphabet", this, database);On an incoming JSON message a new instance of the class AlphabetApi is created, which in turn, creates a new instance of GetLetters class and starts derived task. When the callback DatabaseExecSQLResult is called, the GetLetters instance send a JSON response by calling SendGetLettersResponse function and completes the task.
alphabet_api.h
namespace Appcppmoreexample {
class AlphabetApiContext : public UJsonApiContext {
class JsonApiContext * jsonApiContext;
class IDatabase * database;
char * name;
class AlphabetApi * alphabetApi;
public:
AlphabetApiContext(const char * name, JsonApiContext * jsonApiContext, IDatabase * database);
virtual ~AlphabetApiContext();
class JsonApi * CreateJsonApi(class IJsonApiConnection * connection, class json_io & msg, word base) override;
class JsonApi * JsonApiRequested(class IJsonApiConnection * connection) override;
const char * Name() override;
};
class AlphabetApi : public JsonApi, public UTask {
char * name;
IJsonApiConnection * connection;
IDatabase * database;
ITask * task;
void TaskComplete(class ITask * const task) override;
void TaskFailed(class ITask * const task) override { this->TaskComplete(task); }
public:
AlphabetApi(const char * name, IJsonApiConnection * connection, IDatabase * database);
virtual ~AlphabetApi();
const char * Name() override;
void Message(class json_io & msg, word base, const char * mt, const char * src) override;
void JsonApiConnectionClosed() override;
};
}alphabet_api.cpp:
#include "platform/platform.h"
#include "common/interface/json_api.h"
#include "common/ilib/json.h"
#include "common/interface/task.h"
#include "common/interface/database.h"
#include "alphabet_db_task.h"
#include "alphabet_api.h"
using namespace Appcppmoreexample;
AlphabetApiContext::AlphabetApiContext(const char * name, JsonApiContext * jsonApiContext, IDatabase * database)
{
this->name = _strdup(name);
this->jsonApiContext = jsonApiContext;
this->database = database;
this->jsonApiContext->RegisterJsonApi(this);
}
AlphabetApiContext::~AlphabetApiContext()
{
free(name);
delete this->alphabetApi;
}
class JsonApi * AlphabetApiContext::CreateJsonApi(IJsonApiConnection * connection, json_io & msg, word base)
{
return 0;
}
class JsonApi * AlphabetApiContext::JsonApiRequested(IJsonApiConnection * connection)
{
this->alphabetApi = new AlphabetApi(name, connection, database);
return this->alphabetApi;
}
const char * AlphabetApiContext::Name()
{
return name;
}
AlphabetApi::AlphabetApi(const char * name, IJsonApiConnection * connection, IDatabase * database)
{
this->name = _strdup(name);
this->connection = connection;
this->database = database;
this->task = nullptr;
}
AlphabetApi::~AlphabetApi()
{
free(name);
}
const char * AlphabetApi::Name()
{
return name;
}
void AlphabetApi::Message(json_io & msg, word base, const char * mt, const char * src)
{
if (!strcmp(mt, "GetLetters")) {
bool present = false;
unsigned int limit = msg.get_unsigned(base, "limit", &present);
if (!present || limit > 10 || limit == 0) limit = 10; // max limit 10
ulong64 offset = msg.get_ulong64(base, "more", &present);
if (!present) offset = 0;
this->task = new GetLetters(connection, database, offset, limit, src);
this->task->Start(this);
}
}
void AlphabetApi::TaskComplete(ITask * const task)
{
this->task = nullptr;
delete task;
if (this->connection) {
this->connection->JsonApiMessageComplete();
}
else {
delete this;
}
}
void AlphabetApi::JsonApiConnectionClosed()
{
this->connection = nullptr;
if (this->task) return;
delete this;
}alphabet_db_task.h:
namespace Appcppmoreexample {
class TaskReadAlphabet : public ITask, public UDatabase {
protected:
void DatabaseExecSQLResult(IDatabase * const database, class IDataSet * dataset) override;
void DatabaseError(IDatabase * const database, db_error_t error) override;
virtual void SendGetLettersResponse() = 0;
class IDatabase * database;
ulong64 offset;
unsigned int limit;
class IDataSet * dataset = 0;
public:
TaskReadAlphabet(IDatabase * database, ulong64 offset, unsigned int limit);
~TaskReadAlphabet();
void Start(class UTask * user) override;
};
class GetLetters : public TaskReadAlphabet {
virtual void SendGetLettersResponse() override;
class IJsonApiConnection * connection;
char * src;
public:
GetLetters(class IJsonApiConnection * connection, IDatabase * database, ulong64 offset, unsigned int limit, const char * src);
~GetLetters();
};
}alphabet_db_task.cpp:
#include "platform/platform.h"
#include "common/interface/database.h"
#include "common/interface/task.h"
#include "common/interface/json_api.h"
#include "common/ilib/json.h"
#include "common/lib/tasks_postgresql.h"
#include "common/lib/database_switch.h"
#include "alphabet_db_task.h"
#include "alphabet_api.h"
using namespace Appcppmoreexample;
TaskReadAlphabet::TaskReadAlphabet(IDatabase * database, ulong64 offset, unsigned int limit)
{
this->database = database;
this->offset = offset;
this->limit = limit;
}
TaskReadAlphabet::~TaskReadAlphabet()
{
delete dataset;
}
void TaskReadAlphabet::Start(UTask * user)
{
this->user = user;
// The database table alphabet is initialized in the constructor of TaskDbInit class, but is not filled with data.
// Following statement could be used on the psql console to insert the entries
// (SSH to AP, use password ipapps; 'su -' to root, use pass iplinux; psql -d databasename):
// INSERT INTO alphabet (letter) VALUES ('Alpha'), ('Bravo'), ('Charlie'), ('Delta'), ('Echo'), ('Foxtrot'), ('Golf'),
// ('Hotel'), ('India'), ('Juliet'), ('Kilo'), ('Lima'), ('Mike'), ('November'), ('Oscar'), ('Papa'), ('Quebec'),
// ('Romeo'), ('Sierra'), ('Tango'), ('Uniform'), ('Victor'), ('Whiskey'), ('Xray'), ('Yankee'), ('Zulu');
database->ExecSQL(this, DB_EXEC_FETCH_ALL, "SELECT * FROM alphabet WHERE id>%llu ORDER BY id ASC LIMIT %llu::BIGINT", offset, limit);
}
void TaskReadAlphabet::DatabaseExecSQLResult(IDatabase * const database, IDataSet * dataset)
{
this->dataset = dataset;
this->SendGetLettersResponse();
}
void TaskReadAlphabet::DatabaseError(IDatabase * const database, db_error_t error)
{
Failed();
}
GetLetters::GetLetters(IJsonApiConnection * connection, IDatabase * database, ulong64 offset, unsigned int limit, const char * src)
: TaskReadAlphabet(database, offset, limit)
{
this->connection = connection;
this->src = _strdup(src);
}
GetLetters::~GetLetters()
{
free(src);
}
void GetLetters::SendGetLettersResponse()
{
if (this->connection) {
const char * name = ((class AlphabetApi *)this->user)->Name();
char sb[1000];
char b[300]; // must fit all letters
char * tmp = b;
class json_io send(sb);
word base = send.add_object(0xFFFF, 0);
send.add_string(base, "api", name);
send.add_string(base, "mt", "GetLettersResult");
word letters = 0;
ulong64 more = 0;
if (this->dataset != nullptr) {
while (!this->dataset->Eot()) { // check for the end of table
if (letters == 0) letters = send.add_array(base, "letters"); // add letters array only at the first run and only if dataset is not empty
send.add_printf(letters, 0, tmp, "%s", this->dataset->GetStringValue("letter")); // must use printf, cause calling Next invalidates pointer
more = this->dataset->GetULong64Value("id"); // the value of the last id is stored in the local variable on stack
this->dataset->Next(); // the pointers returned by the dataset Get functions are not more valid after the call of Next
}
}
if (more) send.add_ulong64(base, "more", more, tmp);
if (this->src) send.add_string(base, "src", this->src);
this->connection->JsonApiMessage(send, sb);
}
this->Complete();
}
To create a table "alphabet" when App instance is started, extend the class TaskDbInit by a member class TaskPostgreSQLInitTable initAlphabet.
cppmoreexample_tasks.h:
namespace Appcppmoreexample {
class TaskDbInit : public ITask, public UTask {
class TaskPostgreSQLInitTable initAlphabet;
void TaskComplete(class ITask * const task) override;
void TaskFailed(class ITask * const task) override;
public:
TaskDbInit(IDatabase * database);
virtual ~TaskDbInit();
void Start(class UTask * user) override;
};
}
Add default constructor for member variable and extend the constructor of the TaskDbInit by AddColumn directives:
cppmoreexample_tasks.cpp:
TaskDbInit::TaskDbInit(IDatabase * database)
: initAlphabet(database, "alphabet")
{
initAlphabet.AddColumn("id", "BIGSERIAL PRIMARY KEY NOT NULL");
initAlphabet.AddColumn("letter", "VARCHAR(32) UNIQUE NOT NULL");
}
Add new files to the Makefile.
cppmoreexample.mak:
include cppmoreexample/apps/apps.mak
APP_OBJS += $(OUTDIR)/obj/cppmoreexample.o
$(OUTDIR)/obj/cppmoreexample.o: cppmoreexample/cppmoreexample.cpp $(OUTDIR)/cppmoreexample/cppmoreexample.png
APP_OBJS += $(OUTDIR)/obj/cppmoreexample_tasks.o
$(OUTDIR)/obj/cppmoreexample_tasks.o: cppmoreexample/cppmoreexample_tasks.cpp
APP_OBJS += $(OUTDIR)/obj/alphabet_api.o
$(OUTDIR)/obj/alphabet_api.o: cppmoreexample/alphabet_api.cpp
APP_OBJS += $(OUTDIR)/obj/alphabet_db_task.o
$(OUTDIR)/obj/alphabet_db_task.o: cppmoreexample/alphabet_db_task.cpp
$(OUTDIR)/cppmoreexample/cppmoreexample.png: cppmoreexample/cppmoreexample.png
copy cppmoreexample\cppmoreexample.png $(OUTDIR)\cppmoreexample\cppmoreexample.png