> ## Documentation Index
> Fetch the complete documentation index at: https://help.lobyco.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Reporting by Lobyco

> Reporting delivered by Lobyco.

Getting data in relation to analytics and reconciliation of Lobyco products

To ensure retailers can be up to date how Lobyco products are being used, what campaigns & promotions are running, understanding how much bonus(loyalty bonus) liability has been create and for reconciliation purposes, Lobyco enables retailer to retrieve extensive data from Lobyco services that runs your full Lobyco scope of products.

**Retrieved by Client from Lobyco Data lake utilising Databricks Delta Sharing.**

Recipients can access the shared data using various computing tools and platforms, including Databricks, Apache Spark, Pandas, and Power BI. Delta Sharing offers two sharing modes: Databricks-to-Databricks sharing for sharing data within Databricks accounts and open Delta Sharing for sharing data with users outside of Databricks. Credentials will be shared as the project progresses. The retailer will be expected to retrieve data and ingest to any data warehouse and build out reporting. Lobyco can provide examples on how it can be received depending on client setup, i.e. a Python script to retrieve specific data.

**App analytics / Firebase data**

If your mobile app is powered and delivered by Lobyco, analytics in relation to your app users will be very insightful. In this scenario Lobyco will deliver data in accordance with your Mobile App Scope. The app analytics empowers a retailer to investigate page views per features, avg. time spent on specific pages in the app, features mostly used, peak times etc.

App analytics is available for clients to retrieve by Lobyco Data lake utilizing Databricks Delta Sharing.

## Guide on how to retrieve data by Databricks

Delta Sharing is an open protocol for secure real-time exchange of large datasets, which enables secure data sharing across products for the first time. It is a simple REST protocol that securely shares access to part of a cloud dataset. It leverages modern cloud storage systems, such as S3, ADLS, or GCS, to reliably transfer large datasets.

With Delta Sharing, the user accessing shared data can directly connect to it through Pandas, Tableau, or dozens of other systems that implement the open protocol, without having to deploy a specific platform first. This reduces their access time from months to minutes, and makes life dramatically simpler for data providers who want to reach as many\
users as possible.

The Data Platform still supports the traditional data exchange with files and Storage Accounts, but we now also offer Delta Sharing, an open protocol for data sharing that makes it easier and more secure to share data between organizations.

* Improved security and data governance
* Full and incremental data loads
* Support for larger data sets

More origin Docs about DeltaShare:\
[https://delta.io/sharing/](https://delta.io/sharing/)\
[https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md#overview](https://github.com/delta-io/delta-sharing/blob/main/PROTOCOL.md#overview)

**In this guide, I'll cover two ways to exchange data via delta sharing:**

* **Option 1: Via Python**
* **Option 2: Via API**

## Prerequisites

*Before using Delta Sharing, you will receive a unique URL. From this URL, you can **download a configuration file** that contains all necessary credentials. **Important**: Make sure to **save this file in a secure location**, as it is required for accessing the shared data.*

<Tabs>
  <Tab title="Option 1: Via Python">
    ## Data exploration using Python

    Share\_file\_path : name and location of config file that you have downloaded.

    <img src="https://mintcdn.com/lobyco-4c9fb3ad/wpdZgjvGMgh4Gbhy/images/f39739e8681d09a6b380d2552275bc64b0dee41c.png?fit=max&auto=format&n=wpdZgjvGMgh4Gbhy&q=85&s=2839716fc51d092d1b6ed04d5c867cae" alt="" width="886" height="1299" data-path="images/f39739e8681d09a6b380d2552275bc64b0dee41c.png" />

    <img src="https://mintcdn.com/lobyco-4c9fb3ad/wpdZgjvGMgh4Gbhy/images/6e794bbd1a689b0f9bae061d996eca8ec2761b68.png?fit=max&auto=format&n=wpdZgjvGMgh4Gbhy&q=85&s=7fb2d7a977e636404f5252f13ed37a99" alt="" width="868" height="608" data-path="images/6e794bbd1a689b0f9bae061d996eca8ec2761b68.png" />
  </Tab>

  <Tab title="Option 2: Via API">
    ## Data exploration using API

    <Steps>
      <Step>
        ## Auth 2.0 in Postman

        In config file you will see following structure:

        `{ "shareCredentialsVersion":1, "bearerToken":"TOKEN", "endpoint":"https://DATABRICKS-URL.net", "expirationTime":"9999-12-31T23:59:59.999Z" }`

        <img src="https://mintcdn.com/lobyco-4c9fb3ad/wpdZgjvGMgh4Gbhy/images/8538dee791553998972083c218a600a2e8cb2015.png?fit=max&auto=format&n=wpdZgjvGMgh4Gbhy&q=85&s=175890779da92f1dcb4249b537aedef2" alt="" width="1402" height="510" data-path="images/8538dee791553998972083c218a600a2e8cb2015.png" />
      </Step>

      <Step>
        ## List all Tables in a Share

        This is the API to list all the tables under all schemas in a share.

        **GET : \{prefix}/shares/\{share}/all-tables**\
        \{prefi}x - our databricks url from config file\
        \{share} - name of “share” (got from GET : \{prefix}/shares)

        ### Get name of “share”

        <img src="https://mintcdn.com/lobyco-4c9fb3ad/wpdZgjvGMgh4Gbhy/images/4a0c742a8a0cfc706acda67043640210ac67f6a8.png?fit=max&auto=format&n=wpdZgjvGMgh4Gbhy&q=85&s=9440afc5191994a9c9128da44e683150" alt="" width="1288" height="699" data-path="images/4a0c742a8a0cfc706acda67043640210ac67f6a8.png" />

        ### Get names of all tables shared with you

        <img src="https://mintcdn.com/lobyco-4c9fb3ad/wpdZgjvGMgh4Gbhy/images/30bc683dc2e266a80f754d9b09eb84e12c7b8700.png?fit=max&auto=format&n=wpdZgjvGMgh4Gbhy&q=85&s=d453a911fd1da624fa82e29fb8903cbe" alt="" width="1231" height="714" data-path="images/30bc683dc2e266a80f754d9b09eb84e12c7b8700.png" />

        As we could see, only one table is shared now (table\_name=”stamps”)
      </Step>

      <Step>
        ## Table metadata

        This is the API for clients to query the table schema and other metadata.

        **GET : \{prefix}/shares/\{share}/schemas/\{schema}/tables/\{table}/metadata**

        <img src="https://mintcdn.com/lobyco-4c9fb3ad/wpdZgjvGMgh4Gbhy/images/50bbd4fc5f397753d113af0af1a3b9c56d5674fa.png?fit=max&auto=format&n=wpdZgjvGMgh4Gbhy&q=85&s=4ebee426606bf59b908bf0e18aa19038" alt="" width="1477" height="693" data-path="images/50bbd4fc5f397753d113af0af1a3b9c56d5674fa.png" />
      </Step>

      <Step>
        ## Full load

        This is the API for clients to read data from a table.

        **POST : \{prefix}/shares/\{share}/schemas/\{schema}/tables/\{table}/query**

        <img src="https://mintcdn.com/lobyco-4c9fb3ad/wpdZgjvGMgh4Gbhy/images/d152e1c1194defde6a5a11f2ba787e7288ee2461.png?fit=max&auto=format&n=wpdZgjvGMgh4Gbhy&q=85&s=a7c4745f20a1efe696b3c717edcb5dc3" alt="" width="1484" height="692" data-path="images/d152e1c1194defde6a5a11f2ba787e7288ee2461.png" />

        By reading or downloading file by this URL you could see the data itself.
      </Step>

      <Step>
        ## Incremental load

        The change data feed represents row-level changes between versions of a Delta table. It records change data for UPDATE, DELETE, and MERGE operations. If you leverage the connectors provided by this library to read change data feed, it results in three metadata columns that identify the type of change event, in addition\
        to the data columns:

        * \_change\_type (type: String): There are four values: insert, update\_preimage, update\_postimage, delete. preimage is the value before the update, postimage is the value after the update.
        * \_commit\_version (type: Long): The table version containing the change.
        * \_commit\_timestamp (type: Long): The unix timestamp associated when the commit of the change was created, in milliseconds.

        *You could also schedule a daily cron job to automatically retrieve the file with each incremental version.*

        **GET : \{prefix}/shares/\{share}/schemas/\{schema}/tables/\{table}/changes**

        You could also set up specific start and end version to read only the latest data from these versions:

        <img src="https://mintcdn.com/lobyco-4c9fb3ad/wpdZgjvGMgh4Gbhy/images/34d83b71715c01f2f178800e5c36a6b6e2cbf89f.png?fit=max&auto=format&n=wpdZgjvGMgh4Gbhy&q=85&s=4525dae3bbaaf9a57b483b8c91641965" alt="" width="1488" height="674" data-path="images/34d83b71715c01f2f178800e5c36a6b6e2cbf89f.png" />

        On the screenshot we can observe two URLs for both versions 12 and 13. And we can also read these files in any way we want. Since only changes are loaded, the process is quicker and can be executed more frequently, enabling near real-time updates with minimal disruption to ongoing operations. This is critical for environments where up-to-date data is necessary for decision-making.

        You are also only loading and processing the changes, you consume fewer computational resources and less storage. This directly translates to lower costs, especially in cloud environments where resources are billed based on usage.
      </Step>
    </Steps>
  </Tab>
</Tabs>