Job Title: Sr. Data Integration Engineer
Job Responsibilities:
- Data Integration Pipeline Development:
- Design, build, and manage scalable and reliable data integration pipelines using tools like Boomi, Matillion, and Fivetran.
- Implement and optimize ETL/ELT workflows to ensure efficient data movement between internal and external systems.
- Develop real-time and batch integration processes that meet business requirements while ensuring data accuracy and timeliness.
- Lead the end-to-end development of integration solutions using Boomi, support SDLC during the design process, gathering and documenting business requirements, technical requirements, data mapping, plan and execute regression, integration testing, cutover, go-live, and post go-live activities.
- API and Enterprise System Integration:
- Implement and maintain API-based integrations between various data sources and systems, including Salesforce, Workday, and Aderant.
- Ensure seamless two-way data synchronization between SQL databases and external APIs.
- Work closely with cross-functional teams to integrate API services into the firm’s data architecture.
- Change Data Capture (CDC) & Delta Processing:
- Implement and manage Change Data Capture (CDC) mechanisms across various databases (., SQL, cloud databases) to track and propagate changes in real-time.
- Develop and implement batch and delta processing strategies to optimize data flow, minimize latency, and ensure timely updates.
- Ensure accurate and consistent data capture for historical data tracking and auditing purposes.
- Data Quality and Governance:
- Ensure high data quality across all integration processes, performing regular checks and validations to detect and correct any inconsistencies.
- Collaborate with the data warehouse and analytics teams to ensure data governance policies are followed, with a focus on security and compliance.
- Enterprise Bus Architecture:
- Contribute to designing and implementing the firm’s Enterprise Bus Architecture, ensuring smooth data exchange between various internal and external systems.
- Build and manage processes to propagate real-time changes from SQL tables and other systems to the enterprise bus.
- Ensure the enterprise bus efficiently handles data flows and event-driven processing across the firm’s data ecosystem.
- Performance Optimization:
- Continuously monitor and improve the performance of data integration pipelines to ensure optimal throughput and latency.
- Identify bottlenecks and inefficiencies in existing processes and propose solutions to improve data flow and processing time.
- Collaboration and Communication:
- Collaborate with cross-functional teams to gather requirements, provide updates, and deliver high-quality integration solutions meeting business needs and any reporting requirements.
- Work closely with the data warehouse and analytics teams to ensure the data integration pipelines meet business needs and reporting requirements.
- Collaborate with external vendors and stakeholders to maintain integrations with third-party tools and systems (., Aderant, Intapp, Workday).
- Provide technical documentation and training to other team members to ensure data integration systems' successful operation and maintenance.
Education and Experience Information:
- Required Skills & Experience:
- 7+ years of hands-on experience in data integration and ETL/ELT pipelines.
- Deep experience with data integration platforms, particularly Boomi, Matillion, Fivetran, and DBT (or similar tools).
- Experience with Boomi connectors like SFTP, HTTP Client, DB V2, Azure, etc.
- Strong understanding of API integration and building real-time data synchronization between systems.
- Proven expertise in Change Data Capture (CDC), batch processing, delta processing, and incremental data loading strategies.
- Experience with enterprise bus architecture for managing real-time data flow and event-driven integrations.
- Hands-on experience with SQL databases and cloud-based data warehousing platforms (., Snowflake).
- Proficiency in building data pipelines for systems like Salesforce, Workday, Aderant, and other cloud-based or enterprise applications.
- Strong problem-solving skills and the ability to troubleshoot complex data integration challenges.
- Excellent written and verbal communication skills to work effectively with both technical and non-technical stakeholders.
- Preferred Qualifications:
- Experience with legal domain systems such as Aderant, Intapp Time, Workday, and Compulaw.
- Familiarity with cloud platforms (AWS, Azure, or Google Cloud).
- Understanding of data governance and data security practices within the legal industry.
- Knowledge of data visualization tools (., Tableau, Power BI) and how integrated data supports business reporting and analytics.
- Experience working within a regulated industry or compliance-heavy environment.
- Ability to understand and write any quick SQL and Python scripts.
- In-depth understanding of API design and development, including RESTful and SOAP protocols.
- Familiarity with different data formats, such as JSON, XML, and CSV, and experience with data transformation techniques.
- Solid knowledge of authentication and authorization mechanisms, security protocols, and data privacy considerations in integration projects.
Salary Information:
- For New York, the expected salary range for this position is between $151,400 and $177,000 per year. The actual compensation will be determined based on experience and other factors permitted by law.
Legal Terminology:
- The firm is an equal employment opportunity employer. All qualified applicants will receive consideration for employment without regard to race, national origin, gender, age, religion, disability, sexual orientation, veteran status, marital status, or any other characteristics protected by law.
