Find Jobs
Hire Freelancers

Scrape web page (with pre-tag), store results in MySQL

$30-250 USD

Abgeschlossen
Veröffentlicht vor fast 4 Jahren

$30-250 USD

Bezahlt bei Lieferung
Hi folks! I've got a short web page that needs to be scraped on a regular basis (water level / water flow gauge report in a format that never changes). Values will be stored in MySQL. The data is wrapped in a <pre> tag, so you'll need to write a script that can parse through it (which we can later execute with an hourly cron job). Once the data is stored in MySQL I'll be able to manipulate the data from there. My goal is to generate a chart on my personal website so I can easily see the water level / water flow trends without having to read a bunch of raw data. Eventually I'll set it up so I can receive an email or SMS when the water level gets above/below certain thresholds. You do not need to worry about integrating chart or notification features... I only mention my goal so you'll better understand how the data should be formatted in MySQL. Here's an example page: [login to view URL] - 17 columns (with headers). As you can see there are 24 line items, one for each hour of the day. Your script will probably just need to scrape the entire <pre> tag each time, but it should only insert (1) row in MySQL depending on the hour of the day. For example, your custom "[login to view URL]" script fetches the entire <pre> tag and sorts the data into a "useful" format and your "parse('$add-mysql-row')" function will insert a row based on the time of day. So at 8:00am, the line item at 8:00 gets added to MySQL. At 2:00pm, the line item at 14:00pm gets added to MySQL. That way we're only adding 1 row to MySQL per hour, not all 24. Maybe a switch|case scenario? I'm not a programmer -- just thinking of options. :) Thankfully there are no extra spaces in the title names or values so that "should" make it easier to scrape (maybe?). Perhaps we can use the "SHEF ID" value as the table name, header values as column IDs, and each row would have 17 values. I think Python is probably the most suitable option for scraping data out of <pre> tags (perhaps with the use of Elephant or some other library), but really the language / framework is up to you. This is just a low budget, low priority hobby / project. Nothing fancy. Thanks! ---- I uploaded a sample MySQL file created using the WPDataTables plugin, including 1 row of data. (This plugin can create separate DBs, or connect to existing custom databases, so we're not locked into any specific schema). Very simple structure. ---- Regarding your questions about the chart/graphs. WPDataTables allows me to display the data in "real time" (auto refresh), which is why I opted for that method. Then I can use Zapier /w MySQL to monitor/trigger other actions. That's the main functionality of this project. However, if you want to go a different route (easier? better?), then I am open to adding this data to Google Sheets instead (displaying in chart / text format) as long as it can auto-refresh. Whichever method is easier! :)
Projekt-ID: 26184824

Über das Projekt

21 Vorschläge
Remote Projekt
Aktiv vor 4 Jahren

Möchten Sie etwas Geld verdienen?

Vorteile einer Ausschreibung auf Freelancer

Legen Sie Ihr Budget und Ihren Zeitrahmen fest
Für Ihre Arbeit bezahlt werden
Skizzieren Sie Ihren Vorschlag
Sie können sich kostenlos anmelden und auf Aufträge bieten
Vergeben an:
Avatar des Nutzers
hi sir am interested in your task. myself 6 years experienced developer. my aim to provide quality work to satisfied you. you can check my reviews for assurance of quality task. we can discuss more via chat.
$30 USD in 1 Tag
4,9 (165 Bewertungen)
6,2
6,2
21 Freelancer bieten im Durchschnitt $88 USD für diesen Auftrag
Avatar des Nutzers
************* I had done so many scraping tasks. I am extreme in craping. please give me a chance. *************************** Greetings! My name is Kamlesh Kumar and I am a full-stack web developer. I believe my skills match for your project. I have a good grasp of working with PHP. I see your job description. I am ready for start working right now. I would be easily able to achieve the task. I believe communication is the key to every successful project. I look forward to discussing your project with you. Sincerely, Kamlesh Kumar
$35 USD in 2 Tagen
5,0 (123 Bewertungen)
8,7
8,7
Avatar des Nutzers
I can provide you a simple Python scraper that will output into MySQL in 3-4 hours for just $60. All I need to start is your MySQL schema for testing. Roman
$60 USD in 1 Tag
4,9 (864 Bewertungen)
8,3
8,3
Avatar des Nutzers
Hi sir, This is Lin and I am scraping expert, please check my reivews then you will know. Can we discuss more details then we can start this project asap? Thanks, Lin
$60 USD in 1 Tag
5,0 (590 Bewertungen)
8,4
8,4
Avatar des Nutzers
I am a professional web data scraper specialized using Python, .Net, C#, PHP script. My tool can search data and get information from A to Z with an existing list of English words. Below is the link for your reference as a sample related to my tool being developed. This demo will capture name, address, zip, Email, phone, ratings and reviews in 4 different sites. The final output will be save in *.XLSX format or as your requirement can start as early as possible depending on your approval and acceptance. In relation to this application, I can rest assured I will impart a high quality and reliable, efficient and accurate with the output. Give me a try and I will try to get the best results and finish the project far before the deadline. Thanks,
$100 USD in 3 Tagen
4,9 (217 Bewertungen)
7,3
7,3
Avatar des Nutzers
Hi. I have written many automation projects. Ready to write your project. Will complete within 1 day
$30 USD in 1 Tag
4,9 (234 Bewertungen)
7,0
7,0
Avatar des Nutzers
Hi I'm an expert in web scraping I have read your project description I can build this script in less than 1 day I can do this project for you perfectly I wait for your reply, I can start work now Thanks
$150 USD in 1 Tag
5,0 (43 Bewertungen)
6,6
6,6
Avatar des Nutzers
Hello, I can help you scrape the website. I am ready to start right away Thanks & Regards, Everest Technology
$88 USD in 2 Tagen
5,0 (45 Bewertungen)
5,9
5,9
Avatar des Nutzers
Hello, How are you? Hope you are doing well. I have read all the requirements and ready for discuss. Just message me when you're free. Thanks
$111 USD in 5 Tagen
5,0 (115 Bewertungen)
6,0
6,0
Avatar des Nutzers
⭐⭐⭐⭐⭐ Hello Your requirements: "Scrape web page (with pre-tag), store results in MySQL" I'm a senior PHP developer with 9 years of experience and I'm very familiar with API integration. As you will see in my profile all of my clients are happy with my work and that's why I have 100% job success rate. please contact me and let's discuss more in detail. thanks
$140 USD in 3 Tagen
4,6 (27 Bewertungen)
5,6
5,6
Avatar des Nutzers
Hi, Sir!! ##% Your idea is attracted me to your world. %## i use python selenium and beautiful soup for scrapping @@PLEASE GIVE ME CHANCE TO PROVE MY SKILLS@@ :-):-):-):-) i want to help you with my web scrapping skills:-):-):-) if you HIRE me, i will do my best to complete your project perfectly in time. thanks. Best Regard.
$140 USD in 7 Tagen
5,0 (16 Bewertungen)
5,1
5,1
Avatar des Nutzers
Dear Employer, I have read the project details and confident to work on Scrape Web page I have extensive knowledge on Python Java script Web scraping MySQL Kindly message me so that we can discuss more about the work. Regards Lucky
$200 USD in 3 Tagen
5,0 (34 Bewertungen)
5,2
5,2
Avatar des Nutzers
I'm an experienced software engineer in the Portland area. I've looked at your example page and thorough explanation and I am certain I could complete this for you quickly. I've done quite a few scraping projects lately, and I have done ones which execute via cron and store results in MySQL. Please message me so we can discuss it. Thank you!
$60 USD in 3 Tagen
5,0 (11 Bewertungen)
3,9
3,9
Avatar des Nutzers
Hi There, I am Solution Architect with 12+ years experience of scrapping web applications and developing SharePoint and ASP.NET Applications. I am an expert in scrapping data(Please check my previous reviews) it's 9 years away from freelance so please let me show you my skills and finish the task for you.. Please message me for the details.
$130 USD in 2 Tagen
5,0 (2 Bewertungen)
3,1
3,1
Avatar des Nutzers
hey, I just read your post and feel I can do this for you in minimum time possible and at reasonable price. I have 2 years of experience in web scraping using python, beautiful soup and selenium. I can scrape any kind of data which is present on web. I can assure you, you will get quality work and everything will be done according to your preferences. so, hope to hear from you. regards, Ankit
$35 USD in 1 Tag
4,9 (9 Bewertungen)
2,0
2,0
Avatar des Nutzers
Hi, Hope you are doing good. I have gone through your requirement and I do have skill set you are looking for. I am certified advanced RPA and QA Automation Professional. I have 2+ year of experience with Automation anywhere and successfully delivered 6 bot which are live up and running in production. I also have 10+ year of expertise in Test Automation framewrok designing with Selenium+ java , UFT + VB script and Katalon Studio. I do have very good exposure working with Web,Rest and Soap service, and also desktop based application automation. I also have hands on BDD-Cucumber,Elastic Search,Excel macros, GIT, Gradle,CI CD tools Jenkins,Cloud testing with saucelabs. We can connect and discuss more about your requirements and come up with best solutions. looking forward to working with you. Regards, Anklesh Singh
$110 USD in 2 Tagen
5,0 (1 Bewertung)
1,6
1,6
Avatar des Nutzers
Dear client  Thanks for the  opportunity for your copywriting neends! As a web scraper freelancer I've found many clients enjoy my unquie and high quality of works  with resonable rates. Respectfully Naveen
$30 USD in 6 Tagen
0,0 (0 Bewertungen)
0,0
0,0
Avatar des Nutzers
I have read your post ? no need to worry ? I am here to help you here I would like to to do your work I will make a sql script. I am expert web developer.
$35 USD in 6 Tagen
0,0 (0 Bewertungen)
0,0
0,0

Über den Kunden

Flagge von UNITED STATES
Stillwater, United States
5,0
46
Zahlungsmethode verifiziert
Mitglied seit Jan. 17, 2006

Kundenüberprüfung

Danke! Wir haben Ihnen per E-Mail einen Link geschickt, über den Sie Ihr kostenloses Guthaben anfordern können.
Beim Senden Ihrer E-Mail ist ein Fehler aufgetreten. Bitte versuchen Sie es erneut.
Registrierte Benutzer Veröffentlichte Jobs
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Vorschau wird geladen
Erlaubnis zur Geolokalisierung erteilt.
Ihre Anmeldesitzung ist abgelaufen und Sie wurden abgemeldet. Bitte melden Sie sich erneut an.