![](/_next/image?url=%2Fcase-studies%2Finaki-del-olmo-NIJuEQw0RKg-unsplash.webp&w=3840&q=75)
Databricks DataHub
Rearc designed and implemented a foundational governance pattern, DataHub, supporting sharing of data between internal buisness segments on Databricks.
ASAPP, an Artificial Intelligence product company in NY has been using CI/CD tools such as Github, CircleCI and Terraform to run build and test automation on publicly hosted Github and CircleCI solutions. With a growing customer base, it became essential to improve and enhance the security and compliance posture in their software development and release lifecycle by building and deploying an internal enterprise CircleCI and internal enterprise Github environment. Additionally, in order to improve visibility into software quality, a need for standard code quality and scanning tool became apparent.
We were engaged to build an enterprise CI tooling framework inside the customer’s private AWS environment. Our goal was to build the solution using Terraform with a high degree of automation. The solution should allow engineers to get up and running again quickly in the event of an outage in a fairly automated way.
Our solution was designed to run in a single AWS region with primary HA using snapshot recovery of artifacts and state. The solution was build using various AWS services such as EC2, RDS, S3, Lambda, IAM, SQS, EBS, Route53 etc. The environment was built such that it can be re-instantiated at will in any AWS region by specifying parameters during run-time. Necessary network configurations were set up for various services to communicate with each other. The environment met customer’s internal security and VPC requirements.
The custom CI tooling framework in AWS allowed the customer to quickly migrate their existing applications over to the new setup in order to meet their security and compliance requirements with an improved developer experience.
Read more about the latest and greatest work Rearc has been up to.
Rearc designed and implemented a foundational governance pattern, DataHub, supporting sharing of data between internal buisness segments on Databricks.
Collaborate with partners utilizing Databricks Clean Rooms
Ingest data products from multiple marketplaces into Databricks
Rearc was asked to help a Fortune 500 payments processing company publish data products to the AWS Data Exchange enabling new data-driven revenue streams and broader customer reach.
Tell us more about your custom needs.
We’ll get back to you, really fast
Kick-off meeting