🚫 This project is no longer maintained.
Hey there! So you want to build your own Simple Data Pipe connector? Start here.
This Simple Data Pipe connector template illustrates how to
- define OAuth connectivity for a cloud data source,
- load a static set of data sets (2016 election speeches) and
- store the results in Cloudant.
This sample connector illustrates how to implement OAuth connectvity for a cloud data source, such as reddit. A valid reddit user id is required to create and run a Simple Data pipe that fetches a small data set for illustrative purposes.
Deploy the Simple Data Pipe in Bluemix using the Deploy to Bluemix button or manually.
This connector does not require any additional Bluemix service.
When you follow these steps to install this connector, add the following line to the dependencies list in the package.json
file:
"simple-data-pipe-connector-oauth-template": "git://github.com/ibm-cds-labs/simple-data-pipe-connector-oauth-template.git",
You need to register the Simple Data Pipe application before you can use it to load data.
- Open the reddit web page and log in.
- Click Preferences and select the apps tab.
- Create another app...
- Assign an application name and enter an optional description.
- As redirect URL enter
https://<simple-data-...mybluemix.net>/auth/passport/callback
.
Replace
<simple-data-...mybluemix.net>
with the fully qualified host name of your Simple Data Pipe application on Bluemix.
- Click create app.
- Copy the application id displayed under your application name (e.g. vv5ulJR3...20Q) and the secret (e.g. j60....CFSyAmSY).
To configure and run a pipe
- Open the Simple Data Pipe web console.
- Select Create A New Pipe.
- Select Sample OAuth Data Source for the Type when creating a new pipe
- In the Connect page, enter the application id and secret from the reddit app preferences page.
- Select the data set (or data sets) to be loaded.
- Schedule or run the data pipe now.
Copyright [2016] IBM Cloud Data Services
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.