Hello,
I have extensive experience ingesting on-premises data into Databricks as an Azure expert with a focus on data engineering. Specifically, I handle audit files with sizes ranging from 1GB to 10GB. I guarantee data integrity and performance in Databricks environments by utilizing Delta Tables.
I have experience building reliable pipelines for data intake, optimizing for daily updates, and putting best practices for processing efficiency into practice. I have extensive knowledge of on-premises data transfer protocols and secure connections to Databricks, and I have successfully managed projects of a similar nature using automated ingestion workflows.
Key Competencies:
MS Azure Services.
Developing, deploying, and configuring Azure Web Apps and Azure App Services using Visual Studio; experience in cloud migration and integration.
Azure Active Directory, ADFS, OAuth, and SAML for authentication and authorization.
building solutions using Event Hub, Service Bus, Blob Storage, Azure Files, and Table Storage.
Developing and hosting REST APIs using Azure API Management, Swagger, and integration with Azure web apps.
Designing, hosting, and managing data in SQL Azure, Document DB, Mongo DB, Table Storage, Azure Data Lake, and blob storage.
I'm eager to discuss your project in more detail and provide insights on how I can bring value to your business through my skills.
Best regards,
Imtiyaz