Skip to main content

Connecting to SFTP (Simple File Transfer Protocol)

Overview

SFTP (Simple File Transfer Protocol) features hierarchical folders and file management. It supports user authentication that requires the user to enter username and password to login into the server. You can implement SFTP by opening a TCP connection to the remote host’s port.

SFTP consists of 11 commands and supports features like user access control, file transfers, directory listing, directory changing, file renaming, and deleting. It supports three types of data transmission, namely:

  • American Standard Code for Information Interchange (ASCII): ASCII bytes are taken from the source system file, sent over the connection, and saved in the destination system file.
  • Binary: The 8-bit bytes are extracted from the source system file, sent through the connection, and saved in the target system file.
  • Continuous: The bits are taken from the original system file and transmitted over the network in 8-bit bytes without respecting word boundaries. Without any word boundaries, the bits are continuously received by the target system.

Prerequisite

You will need the following details to configure and import data:

  • Host
  • Port
  • Host Key Fingerprint
  • Username
  • Password
  • Storage path.

Limitation

  • Simple File Transfer Protocol (SFTP), which uses the Transmission Control Protocol port number 115, is an unencrypted, simplified form of File Transfer Protocol (FTP). It is less powerful than FTP and contains certain helpful features that are missing from Trivial FTP (TFTP).

Import data using SFTP connector

Follow the below steps to create a new dataflow for the SFTP import connector:

  1. Go to Dataflow > Imports.
  2. Click New dataflow.

The Set dataflow name page appears.

Alt text

  1. In the Set dataflow name page, type dataflow name in the Name text area.
  2. Click Next.

The Choose connector page appears.

Alt text

To add SFTP connector

  1. In the Choose connector page, select SFTP connector.
note

You can also use the Search feature to find the connector. The SFTP connector is located under the Marketing category.

Alt text

  1. Enter Display Name for your dataflow in the text area.
  2. Enter Description for your dataflow in the text area.
  3. Click Next.

The Connect to SFTP page appears.

Alt text

To configure SFTP

Follow the below steps to configure the connection to SFTP:

  1. Enter your credentials such as Host, Port, Host Key Fingerprint, User Name and Password to connect with SFTP.

  2. Click the folder icon on Storage Path to select the storage path folder.

Once you select your Storage path, the Table Details columns appear.

Alt text

  1. Enter the Table details to process the data.
ItemDescription
PurposeOption to assign a purpose (Data or Metadata) for each table.
Data
Loads customer data.
Metadata
Loads Metadata.
File NameDisplays the name of the file that you imported.
Table NameDisplays the imported table name.
Datetime formatDisplays a number of Datetime Formats and Skypoint AI is set to automatically detect them.
DelimiterDisplays available separators for the variables in the imported data.
First Row as HeaderCheck the box for the system to automatically collect the data according to the Header Contents.
Advanced SettingsSelect the options to modify the default settings. It gives more flexibility to apply advanced use cases.
  1. If necessary, select the Advance settings option to modify the default settings.

The Advanced settings pop-up appears.

Alt text

If you want toThen
Modify data types such as fixed or variable data types.Select from the Compression type. It allows you to reduce the size of data by removing the number of bits.
Change the delimiterClick Row delimiter. By default, a column delimiter is selected, and each row is separated with a comma.
Change information or instructionChoose from the Encoding list. By default, UTF-8 encoding is selected.
Modify the escape character such as backslash (\) or slash (/)Select from the Escape character.
Apply different quote characters such as Single quote (') or Double quote (").Select from the Quote character.
  1. Click Save to apply the changes.

Run, edit, and delete the imported data

After saving the connection, imported data appears on the Dataflow page. Also, you can see a list of the created tables in the Databases section.

Alt text

ItemDescription
NameDisplays the name of the imported Dataflow.
TypeDisplays connector type symbol.
StatusIndicates whether the data is imported successfully.
Tables CountDisplays the number of tables.
Created DateDisplays date of creation.
Last refresh typeDisplays the refresh value. You can see if the value is Full or Incremental after the last data refresh.
Updated DateDisplays last modified date.
Last RefreshDisplays the latest refresh date, which updates each time you refresh the data.
Group byOption to view the items in a specific Group (For example, name, type, status).
  • Select the horizontal ellipsis in the Actions column and do the following:
If you want toThen
Modify the DataflowSelect Edit and modify the Dataflow. Click Save to apply your changes.
Execute the DataflowSelect Run.
Bring the data to its previous stateSelect Rollback.
Delete the DataflowSelect Remove and then click the Delete button. All tables in the data source get deleted.
See the run history of the DataflowSelect Run history.

Next step

After completing the data import, start the Master Data Management (MDM) - Stitch process to develop a unified view of your customers.