I need to develop a software in JS and nodeJS that can capture data from cookies in local user storage and put it in a database.
I'll provide a list of e-commerces and cookie names to look for and pick these data.
Data to be collected:
1 - After mapping the cookie and collect the data, send it to the server encrypted, and before store in a database, decryipt.
2- This JS must be encrypted before loaded in the client browser.
3 - Use a queue system like sidekiq, resque, rabbitmq,etc… this is important to reply the http request quickly and the user don’t need to wait the data storage process finish.
Example of website to collect data:
[url removed, login to view]
Cookies: neemuDataUserEmail / neemuDataUserName /
Step 2 - Collect rules ( NodeJS prepare the JS based on referer )
Setup of time range to collect this data: hours / minutes / seconds
Setup of geo to collect, adding geos to NOT COLLECT
Setup of exclusion IP Addresses
Flag user to don't collect again in X hours
Restrict one capture per IP Adress
Step 3 - Administrative Tools ( backend with nodeJS )
Add domains as blacklist emails
Add full email addresses
Import CSV file with blacklisted emails
Match URLs visited (step 1) with database of URLs collected in other tool, and update user register with category ID in 2 levels (ie: Electronics / Mobile Phones)
Create Segmenting Tool using as filters:
E-commerce where data was collected
Category / Subcategory
Export Filter in XLS / CSV
List of campaigns with tabled data