"g3"
For the past 6 months we've been working on a complete redesign of our identity and access management platform from the ground up.
This represents a completely new 3rd generation of our platform - "g3". Our new platform is faster, more stable, highly efficient, simpler to deploy, easier to debug and fix, quicker to scale, easier to develop, has much better integrated testing, has significantly better logging, better caching and cache invalidation, and has a much smarter and more consistent software architecture. IAM Cloud is now fully deployed on the latest .NET Framework.
In other words - lots has changed. While the cloud typically facilitates 1-2 week sprints with gradual incremental advancements, we have had no choice in this case but to undertake a major project to get the new platform released in major jumps. First we had to release a whole new API, then a new password service, and latest a radically improved authentication service. We're now working on the final phase which is an upgrade to our identity sync engine and agent, which we're expecting to release within the next 4-6 weeks. Below is a summary of what we've accomplished so far in our API, Password and Authentication releases.
Our objectives
1) Shifting our platform away from Azure IaaS and towards Azure PaaS & micro-services to make IAM Cloud more robust, scalable and cost-efficient to run. IaaS provides a simple way to step into the cloud - and in the early days of Azure it was much more mature than the PaaS technology. However, as Microsoft has continued to improve Azure, it is no longer tenable for a platform as large as ours to be heavily reliant on IaaS. It's too slow to scale and it's too expensive to run.
2) Platform g3
Since shortly after the inception of IAM Cloud, in large part due to choices made regarding IaaS, our platform became fragmented. Different customers were using different versions of our technology. Migrating early customers who were using the oldest versions of our platform on to the latest stacks was not an insignificant task. We ended up in a place where we were supporting several platforms at once, which made managing our customer environments MUCH more complicated for our support team. It made bugs more difficult to fix. It made critical updates slower to develop and release. "g3" - which isn't a formal name just our way of describing the update - is our concept for having a single unified platform. The last platform we'll ever make, but one that will gradually advance, develop and grow over time to continue supporting our customers' identity and access requirements long into the future.
3) Futureproof
Ok, nothing is futureproof. But we wanted to design a platform that makes intelligent use of the very latest cloud technologies to ensure we're perfectly poised for the next 5+ years of operations. The platform should scale quickly, be easy to debug, quick & simple to release, with complete automation and test integration, full interoperability and extremely resilient, and where possible built using frameworks rather than hardcoded architecture. And finally it should be consistently designed and implemented. This is what we've done.
Infrastructure Upgrades
Full Automation
We've hired a fantastic automation specialist and put automation at the heart of our whole infrastructure, release management and testing processes. Now we can fully deploy our authentication service in a single click. That's just the beginning.
Interoperability & Resilience
We had some unpleasant bumps to our authentication service in Summer 2016. While the root cause was linked to upgrades Microsoft did to its Azure data center networking - the reality is that we needed IAM Cloud to be able to absorb the bumps without causing issues for our customers. One of the main infrastructure changes we made was a transition to full interoperability. In other words, we removed the coupling between different internal IAM Cloud services. Tightly coupling cloud services can give you some nice 'fair-weather' performance benefits, but they're also very fragile - even small delays between data transmissions can very quickly escalate and lead to service failures. Since we made the transition to interoperability, our infrastructure has been rock solid.
Logging and Monitoring
We've spent a considerable amount of time getting total log coverage throughout our whole platform. This is great for internal diagnostics, improved security and threat analysis, and (coming soon) for detailed service reports for customers. We've also got some exciting developments on the way customers will be able to integrate our logging through webhooks and external APIs. We're already doing this internally for our own purposes, but will be releasing it to customers in the coming months. Webhooks provide incredible extensibility of our notifications and monitoring systems to be able to integrate into pretty much any other modern web/cloud based system. We're excited, and you should be too, the possibilities are pretty incredible. Every time we get a notification about some service-event through into one of our private company Slack channels, automation platform Zapier, or marketing automation system Hubspot, we all get a bit giddy.
Architecture Upgrades
Azure Functions
In the right situations, Azure's Functions as a Service technology can be absolutely phenomenal. We're now leveraging it for a lot of our queue based systems. Azure Functions allow truly insane levels of scaling, and have allowed us adopt some of Azure's latest technology standards. Hey, why reinvent the wheel when Microsoft has made a rocket you can tap into!?
Micro-services
Micro-service architecture - specifically Service Fabric in Azure - is a brilliant way to intelligently scale parts of our service independent of others. This enables intelligent performance improvement based on machine learning, massive almost-instant process scaling, and great efficiency as only the components of the platform that need scaling actually scale-up.
Live Debug & . NET Framework
Migrating our whole software stack into the latest .NET Framework (4.5) has unlocked the 'live debug' feature which is like magic. In combination with being able to do fully automated testing and releases, live debug gives us the incredible ability to identity issues in the live environment and then fix them immediately without having to run-up a test environment, try to reproduce the issue, identity the cause, code-fix and then go through a lengthy release process. Bugs can be identified and safely fixed in live in under 5 minutes now. Contrast that to prior to the live debug and release automation, where even the simplest bug fix would take 6 hours before we could get the fix out to the live environment.
Bugs Fixes & Functionality Upgrades
A short summary of the approx 175 improvements & fixes we've made...
- Issues with Single Sign-On following a Microsoft update with Office Apps
- All Known Password Reset issues resolved including inability to reset passwords from the portal
- samAccountName to be used with all password services
- Customized links for password resets are now correctly working
- Generation 2 Identity customers can now authenticate to our Generation 3 Authentication Service.
- Single Sign-On issues with Generation 3 Authentication are resolved
- Password Reset via the Portal fully operational when password complexity is enabled
What's next?
The final stage of our massive "g3" is our identity sync engine. This release will fix some issues certain customers are experiencing in syncing and provisioning. Here are some of the fixes included in the new release:
- Multi valued attributes aren't reliably syncing to Office 365 within our sync-SLA including group membership and proxy addresses
- Changes in single valued attributes are not always being reflected in Office 365 within our sync-SLA including names changes, email address changes
- In new account creation, single-valued and multi-valued attributes are not always syncing correctly to Office 365 and IAM Cloud's Cloud Vault
- Deleted or declassified users are not always being deleted correctly in Office 365
These issues are all currently being handled manually by our support team - so if you experience any of them before the release, make sure you contact support@iamcloud.com or visit https://support.iamcloud.com and log a ticket.
Once this upgrade is complete the whole g3 platform from the APIs, Password Service, Logging System, Identity Engine, Core Infrastructure and Authentication Services will be globally live, and an exciting new era of IAM Cloud will begin!