Find Jobs
Hire Freelancers

Big Data Processing AWS EMR or Redshift

$250-750 AUD

Geschlossen
Veröffentlicht vor etwa 8 Jahren

$250-750 AUD

Bezahlt bei Lieferung
Hi All, Thanks for taking time to bid on the project. I have large amount of log file data that I need to analyse. This data is stored on AWS S3 in .gz txt files that are tab delimited . It contains the following fields (some optional) TIMESTAMP UID GEO URL CATEGORIES USERAGENT META_KEYWORDS KEY_TERMS ENTITIES Sample file is attached - File sizes are from KB to 10 MB. Requirement: 1: To load and analyse the data (Via EMR or Redshift on AWS), this choice is based on keeping costs lowest. Performance is not the main criteria. 2: Calculate high level metrics (By time period) including: A: Domain Name based counts B: Domain to Key Terms frequency C: Useragent frequencies D: Entities Frequencies E: Categories Frequencies F: List of Domains based on Categories G: list of Domains based on Key Terms Please ask questions before you bid not after. I am open to suggestions. Regards Happy Bidding
Projekt-ID: 9406922

Über das Projekt

9 Vorschläge
Remote Projekt
Aktiv vor 8 Jahren

Möchten Sie etwas Geld verdienen?

Vorteile einer Ausschreibung auf Freelancer

Legen Sie Ihr Budget und Ihren Zeitrahmen fest
Für Ihre Arbeit bezahlt werden
Skizzieren Sie Ihren Vorschlag
Sie können sich kostenlos anmelden und auf Aufträge bieten
9 Freelancer bieten im Durchschnitt $827 AUD für diesen Auftrag
Avatar des Nutzers
Hi. How are you? what need you do with this data? maybe i can put on topics to apache kafa (a queue services with data persistence) and make micro services to route to destiny of data. Is ok?
$1.111 AUD in 5 Tagen
0,0 (0 Bewertungen)
0,0
0,0
Avatar des Nutzers
Hello! Can do this task for you very quickly. Have experience using Amazon EMR in old project. I have wide experience in writing utilities on C++/C#/Python/R/PHP (including client-servers scripts, web scraping, working with databases, monitoring and control systems, and so on). May start right now. Almost always online, waiting for your answer Thank you.
$650 AUD in 5 Tagen
0,0 (0 Bewertungen)
0,0
0,0
Avatar des Nutzers
Hi, I have some questions regarding the timeframe and others for this project. Although you've mentioned that performance is no the main criteria, what's your worst case scenario in terms of time for analysis of a 10 MB file and what would be the instance specifications on AWS or Redshift that we'd be working on ?
$700 AUD in 7 Tagen
0,0 (0 Bewertungen)
0,0
0,0
Avatar des Nutzers
we have a skilled team of machine learning and data mining experts. we have completed several project involving clustering, feature space reduction using algorithms like PCA and data analysis using python, R and Matlab. Our team can help you with this project. Please share more details so we can talk further. final offer and timeline will be decided after discussing the details.
$1.000 AUD in 10 Tagen
0,0 (0 Bewertungen)
0,0
0,0
Avatar des Nutzers
Hi Team, I am having 4+ years of experience in data analytic and served 15+ clients. As a suggestion : This work could be done using Elasticsearch / Logstash and Kibana. Where reports and dashboard can be generated using Kibana for the mentioned requirement as below : 1: To load and analyse the data (Via EMR or Redshift on AWS), this choice is based on keeping costs lowest. Performance is not the main criteria. : I would suggest to use ELK stack nothing but Elasticsearch , Logstash and Kibana which is open source and can be integrated on AWS 2: Calculate high level metrics (By time period) including: Graph can be plotted to demonstrate the same (for all below metrics). A: Domain Name based counts B: Domain to Key Terms frequency C: Useragent frequencies D: Entities Frequencies E: Categories Frequencies F: List of Domains based on Categories G: list of Domains based on Key Terms Let me know if we can discuss for the same and start ASAP. Also if you want a demo just give me few data say 100 entries , I will do it manually in my environment and come up with a small demo. (One portfolio is attached in my profile as well which is having analysis of my Gmail Data) If you are thinking I do not have any experience on Freelancing or projects so i would suggest you to check my Upwork profile for the work i have done and my portfolio as well, As started bidding on freelancing recently so no portfolio as such.
$727 AUD in 10 Tagen
0,0 (0 Bewertungen)
0,0
0,0

Über den Kunden

Flagge von AUSTRALIA
Australia
4,9
67
Zahlungsmethode verifiziert
Mitglied seit Sept. 3, 2003

Kundenüberprüfung

Danke! Wir haben Ihnen per E-Mail einen Link geschickt, über den Sie Ihr kostenloses Guthaben anfordern können.
Beim Senden Ihrer E-Mail ist ein Fehler aufgetreten. Bitte versuchen Sie es erneut.
Registrierte Benutzer Veröffentlichte Jobs
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Vorschau wird geladen
Erlaubnis zur Geolokalisierung erteilt.
Ihre Anmeldesitzung ist abgelaufen und Sie wurden abgemeldet. Bitte melden Sie sich erneut an.