Job Opportunity at Sonata Software: Digital Architect (SSL_DL_D&A MS_DA)
Join Sonata Software as a Digital Architect to leverage your expertise in data architecture, integration, and optimization. Apply your skills in a dynamic environment to drive innovation and efficiency in data management solutions.
Location: BG4, Bangalore, Karnataka, India
Department: Data & Analytics MS
Job Posted On: Jul 26, 2024
Employee Type: Permanent
Experience Range: 0 - N.A.
Job Description: Data Architect
Key Responsibilities:
Data Integration and Ingestion:
- Design, develop, and manage data pipelines for ingesting data from heterogeneous systems into Fabric Lake.
- Implement ETL (Extract, Transform, Load) processes to ensure data is accurately and efficiently transferred, transformed, and stored.
Data Modeling and Architecture:
- Develop and maintain data models that support business requirements.
- Ensure data architecture aligns with best practices and supports scalability, performance, and security.
- Utilize Spark for data transformation, conversion, and storing/retrieving data from one lake.
- Build and manage scalable data warehouses in Azure Synapse Analytics, optimizing for performance and cost.
- Implement partitioning, indexing, and clustering strategies to enhance query performance.
Data Processing and Transformation:
- Utilize tools and services within Microsoft Fabric to process and transform data as needed.
- Implement data cleansing and transformation processes to ensure data quality and consistency.
Data Storage and Management:
- Manage and optimize data storage solutions within Microsoft Fabric, including data lakes, databases, and warehouses.
- Implement data governance and management practices to ensure data integrity and compliance.
Performance Optimization:
- Monitor and optimize the performance of data pipelines and storage solutions.
- Identify and address performance bottlenecks and ensure efficient data processing.
Security and Compliance:
- Implement security measures to protect data within Microsoft Fabric.
- Ensure compliance with relevant data protection regulations and corporate policies.
Collaboration and Communication:
- Work closely with data analysts, data scientists, and other stakeholders to understand data requirements and deliver solutions.
- Communicate effectively with team members and stakeholders to ensure alignment and address any issues.
Troubleshooting and Support:
- Provide support for data-related issues and troubleshoot problems within the data environment.
- Monitor data workflows and resolve any issues to ensure continuous data availability.
Automation and Innovation:
- Explore opportunities for automating data processes to increase efficiency.
- Stay updated with the latest trends and advancements in data engineering and integrate new technologies and methods as appropriate.
Data Governance and Lineage:
- Implement and manage data governance policies to ensure proper data stewardship.
- Utilize lineage tracking features to provide visibility into data flow and transformations.
Required Tools and Technologies:
- Azure Synapse Analytics: For data integration, warehousing, and analytics.
- Synapse Studio: For developing and managing Synapse resources.
- Azure Data Factory: For orchestrating ETL/ELT processes.
- Azure Data Lake Storage: For scalable storage of big data.
- Power BI: For data visualization and reporting.
- SQL, Spark SQL, and PySpark: For querying and processing data.
- Azure DevOps: For CI/CD pipelines and source control.