Datenarchitektur Jobs

My existing Power BI report pulls several large tables from Microsoft Dynamics 365 Business Central through the standard API. The model works, but the scheduled refresh now fails with frequent time-outs; reliability is my biggest concern. So far I have: • enabled incremental refresh on the largest tables • tuned a handful of queries • experimented with splitting data into smaller API calls None of these measures has eliminated the failure. I need an expert who can: 1. pinpoint the exact bottleneck within the current data model, gateway settings, or API calls 2. redesign the retrieval strategy so the report finishes every refresh window without timing out 3. leave me with a clean, documented PBIX (or dataflow) that follows best practice for query folding, partitioning, and ...

We are looking for a Senior Java Backend Developer with deep expertise in high-performance data architectures to join our team. As we scale our career pathway platform, your primary focus will be optimizing how we store, retrieve, and search complex datasets. You will be responsible for bridging our robust Java/Spring services with a modern data stack centered around MongoDB and Elasticsearch to provide our users with lightning-fast, relevant results. Key Responsibilities * Data Architecture: Design and implement scalable data models in MongoDB, ensuring efficient document structures for high-speed read/write operations. * Search Optimization: Architect and maintain Elasticsearch clusters, including index management, custom mapping, and complex query DSL optimization to power advanced s...

I have a rough system architecture sketch that urgently needs a polish so it clearly shows how our database layer can be tuned for better performance. The core of the exercise is to revisit the data flows, pinpoint latency hotspots, and illustrate an optimised structure—caching tiers, read/write segregation, indexing strategies—using a clean, professional diagram. You will start from my existing file (or import it into Lucidchart, Visio, or another tool you’re comfortable with) and reshape it into a concise, single-page system architecture diagram. The spotlight is entirely on the database component; server infrastructure and application logic can be represented at a high level only where they touch the data layer. Deliverables • Editable source file (, Lucidcha...

I’m upgrading our analytics stack and need an expert who can own the Hadoop side and turn raw, high-volume feeds into analysis-ready datasets. The core objective is to design and build end-to-end data pipelines on a Hadoop cluster—this is where I believe Hadoop will be most valuable for the project. Here’s what I need from you: • An architecture that takes terabyte-scale log files, lands them in HDFS, applies basic cleansing, and outputs partitioned Parquet tables queryable from Hive or Spark • All scripts, configs, and scheduling (Oozie, Airflow, or your preferred orchestrator) committed to Git with clear documentation • A deployment guide plus a brief hand-over session so I can reproduce the setup on another cluster Acceptance criteria: the pipeline...

I have an Apache NiFi deployment that has grown organically and now needs a fresh, expert set of eyes. I’d like you to walk through the current flows end-to-end, scrutinising the data-flow design, performance optimisation choices, and the way we handle errors. The ultimate goal is to confirm that the platform is genuinely robust and reliable under present and future workloads. After the review, I’m expecting a detailed report that not only flags every weakness you find but also lays out clear, prioritised recommendations for improvement. Deliverables I will use to sign-off: • A written report (PDF or Markdown) covering architecture observations, specific processor-level findings, and configuration notes • Prioritised recommendations with estimated effort...

Project Objective We are building a complete reporting environment that connects our operational SQL database to a clean, intuitive Power BI reporting suite designed for two primary audiences: 1. Executive Leadership – high-level commercial insights and company performance 2. Operational Managers – actionable operational metrics and drill-downs The goal is to transform raw operational data into clear, reliable decision-making dashboards that update automatically and can be trusted across the organization. The final solution should deliver real-time business intelligence including revenue performance, operational activity, and commercial trends. ⸻ Data Environment Source System SQL Server operational database containing: • Sales transactions • Customers &bu...

I need a data engineer to implement liquid clustering with z-ordering to enhance performance. This project is critical for optimizing our data processing and retrieval speeds. Key Requirements: - Expertise in data engineering - Strong understanding of z-ordering and liquid clustering - Experience in performance optimization techniques Ideal Skills: - Proficiency in SQL and relevant data processing frameworks - Background in managing large datasets - Ability to work independently and meet deadlines Please provide examples of similar work and your approach to the task.

We are looking for a skilled professional with strong experience in healthcare data systems and data modeling concepts. The ideal candidate should have hands-on experience working with Electronic Health Records (EHR) data and a good understanding of clinical and Revenue Cycle Management (RCM) domains within hospital systems. Key Responsibilities •⁠ ⁠Work with healthcare data from leading EHR platforms such as EPIC, Oracle Cerner, MEDITECH, or other top-tier EHR systems. •⁠ ⁠Understand and model clinical and RCM entities, including relationships within hospital and healthcare ecosystems. •⁠ ⁠Apply Kimball modeling techniques and core data modeling concepts for analytics and reporting solutions. •⁠ ⁠Support data integration, transformation, and structuring for health...

PROJECT OVERVIEW We need a freelance engineer to help design and build core components of a prediction markets platform. The focus is correctness, consistency under load, and clean data models that support both UI and analytics. SCOPE OF WORK / RESPONSIBILITIES 1. Prediction market mechanics - YES/NO outcome tokens and implied probability mapping (price ≈ probability) - Liquidity/spreads via order book and/or AMM-style curve - Market lifecycle: open → trading → resolution → settlement - Resolution/oracle inputs, dispute windows, final payout logic 2. Backend + API development - Market discovery/browsing endpoints - Real-time pricing/market updates (WebSockets and/or streaming) - Order placement and cancellation - Idempotent order/trade submission (idempotency keys, re...

I’m building the first version of a mobile app both iOS and Andriod entirely in FlutterFlow, backed by Firebase, and I’d like an expert who can set up a solid, no-code foundation that we can keep expanding. Current scope • User onboarding with Firebase phone-number authentication only (no email or social logins for now). • In-app purchase flow powered by UPI. The app must support both subscription plans and one-time premium unlocks, all handled inside FlutterFlow and recorded in Cloud Firestore/Realtime DB for entitlement checks. • Clean data architecture in Firebase so future modules can be added without painful refactors. Deliverables 1. A ready-to-publish FlutterFlow project with UPI payment widgets and verified test purchases for subscriptions an...

Project Description I am looking for a developer to build a web-based multi-vendor catalog platform where vendors can upload product catalogs and customers can access those catalogs through API authentication similar to PunchOut integrations used in procurement/ERP systems. The platform will allow vendors to manage their catalogs, while customers (ERP systems) can securely access products belonging only to the assigned vendor. The system should include the following components. 1. Vendor Catalog Dashboard The platform should include a secure vendor dashboard where vendors can upload and manage their catalog items. Vendor Access ✔ Each vendor should have unique login credentials ✔ Vendors should only be able to access and manage their own catalog ✔ Admin should be able to create unli...