WMG Data Factory Prod OneLive

WMG Data Factory Prod OneLive runs on a Heroku app. To gain access to the source code an account must be a collaborator to the app though a Heroku account. Open Terminal and perform the following:
  • heroku login
  • heroku git:clone -a YOUR_APP_NAME
  • cd YOUR_APP_NAME
  • git add .
  • git commit -am "make it better"
  • git push heroku master
The app uses the free add-on Heroku Scheduler to run the it daily at 3:00am EST. SysCompany stores the credentials and url for the company of the sales to pull from. S3Creds stores the credentials for the Amazon S3 Bucket the sales will be sent to. titles stores the tab delimited column names of the .txt file being sent. order stores the values of the columns belonging to titles. As a new title is added to titles, it will also need to be added to order and vice versa. Using php curl commands the api access token is retrieved and used in all other curl commands to fetch api data. API structure can be found here. The sales are queried and individually broken down to then gather more specific information on each particular sale using the curl commands. This information is put into an order object and pushed to the allOrders array. The allOrders array is written into a .txt file that is tab delimited which is sent to the S3 Bucket.

Leave a comment