Welcome to BJSSinfo@bjss.com

BJSS Capabilities: Cloud

The Future is in the Cloud. 

Public Cloud applications, platforms and infrastructure can dramatically increase agility and cut costs. But Cloud technology is still evolving, creating concern amongst organisations that their business-critical data is safe and that the rate of business change is properly supported. As they guide their organisations towards the Cloud, CIOs need to address to these concerns. They also need to engage a partner with the expertise and practical experience of delivering at-scale Cloud solutions.

BJSS has helped public and private sector organisations to build their internal capabilities. 

The BJSS Cloud service is uniquely structured for:

  • Identifying and understanding where the client is in the journey to Cloud
  • Helping the client to implement strategies to address the cultural adoption challenge
  • Defining business outcomes and developing strategies to avoid technical debt
  • Running the legacy estate and providing a roadmap to transition to Cloud
  • Helping the client to select the correct applications, data and candidates for Cloud
  • Embedding Agile delivery methods for rapid delivery
  • Integratiing and managing applications through a DevOps approach
  • Tackling the most complex programme tasks including migration, software development, and Cloud-native development such as Microservices and API services.

Optimising the infrastructure of NHS Digital’s Electronice Referral System (e-RS)

Read the Case Study

e-RS was hosted on a mixed Node 4 (co-located physical data centre) and Rackspace infrastructure. An optimisation review identified that the bult infrastructure was not scalable, that ’clean’ build servers could not be repeatedly deployed in a timely fashion, and that the infrastructure was not ’self-healing’. The review also highligted that build times could be reduced with more powerful servers.

BJSS proposed migrating all application build/deployment services to AWS. This included Jenkins build master and slave(s), Selenium grid, SonarQube test environment, Crucible/Fisheye code review tools, Git code repository, Nexus artefact repository and Puppet configuration management.

The infrastructure was designed with scalability and elasticity in mind, with a view to enabling deployment of full staging environments and accommodating future requirements. An important aspect of the engagements was to have all the infrastructure documented as code, using Terraform, to facilitate tracking of the state of the infrastructure, recreating environments as part of Disaster Recovery, and auditing deployed infrastructure.

All infrastructure deployments take place from a centralised Jenkins build server to ensure that important information is held centrally, and to prohibit the ad-hoc deployment of infrastructure not defined in code. As with all e-RS application code in e-RS, deployment code is peer-reviewed and undergoes requisite testing.

Security is integrated using AWS features such as Network Access Control Lists, Identity and Access Management, Security Groups and Encryption, ensuring that data is secure and access rights are correctly controlled.

The BJSS solution uses the following AWS services:

CloudTrail Logs all API Calls, providing an audit trail for all deployment activities.
CloudWatch Monitors several metrics to ensure only authorised security changes are applied.
EBS Provides file storage for Git instances; should the instance terminate a new instance is provisioned and automatically attaches/mounts the volume.
EC2 On-demand EC2 instances combined with Auto-Scaling Group policies provide scalability and elasticity.
IAM IAM roles enforce security policy, ensuring only authorised DevOps staff can deploy/release code.
NACL Control access between public and private subnets, enabling deployment of publicly accessible “Bastion” hosts whilst protecting services in private networks.
RDS The back-end data repository for all services. The solution takes advantage of features such as mult-az high availability to meet project SLAs.
Route53 Enabled simplified and more unified code..
S3  Used to store backup snapshots, Jenkins build histories, and log files
Security Groups Provide an additional layer of fine grain security controls by restricting network traffic between specific services on specific ports.
VPC Provides security and network isolation.

Third party applications and solutions were also used. Terraform allows for the coding of the AWS infrastructure, enabling infrastructure to be easily replicated, moved in the case of a failure, and tracked. Terraform reduced the effort required and ensuring consistent quality of code across the organisation. Also, Packer creates AWS machine images (AMIs) and allows for the coding configuration. While Terraform is responsible for creating an instance (virtual machine) from the AMI, and Puppet is responsible for deploying the application, Packer is responsible for getting the machine image to a state where Puppet can be run.

The successful delivery achieved:

  • Full application stack build time reduced by 50%
  • A reduction of 50% in full application stack build time
  • A 30% reduction in operational support demand %
  • Reductions in infrastructure costs by ~48%
  • Consistent builds
  • Confidence established for migration of further applications into the Cloud

Realise value as early as possible

To establish confidence in the AWS platform, and the reputation for accelerated deployment, it was important to demonstrate the migration of one service as quickly as possible without disruption to the DevOps team. This was achieved by the end of the second (two week) Sprint.

Focus on outcomes

The primary driver was reduced build time. Rather than spending time attempting to accurately size build slaves, with the associated risk of under-specifying, the initial build employed the largest “C” class EC2 instances and retrospectively optimised once performance baselines were established.

BJSS supports each stage of the journey. 

From an initial assessment that identifies opportunities to deploy Cloud technologies, to developing and running the service, the BJSS offering is completely modular. Clients can choose the individual components they require support with, or an end-to-end solution. Our framework considers the cultural impact on an organisation and can be adapted as part of a maturity activity as required.
Assessment and Advisory Services Cloud Readiness Assessment Cloud PoC Digital Business Activation
Architecture Design Services Software Development Cloud Build & Delivery Migration Services
Cost Management and Optimisation API Integration Continuous Integration and Deployment DevOps
Digital Operations Security Management Cloud Competency Centre Cloud Governance
AI Big Data Mobile IoT

Platform Engineering expertise to support a new DVSA Search solution, initially supporting Vehicle Searches, such as Technical Records, Encounters, Operator details and Test History. BJSS Technical Support Services (TSS), integrated and worked collaboratively with the project team to implement this new solution.

Read the Case Study

The delivery included the design and production of a release pipeline, technical consultancy on the use of AWS technologies and integration with legacy systems, support and development of automated infrastructure deployment, providing a reliable method of uplifting the generated data on a nightly basis from the remote location to the RDS instance. The application is a HTML5 Web Application and a hybrid iOS application for corporate mobile devices.

BJSS based the solution on an almost entirely Serverless/PAAS stack within AWS, with Terraform managing infrastructure provision. The use of Serverless technologies continued BJSS’ work with DVSA to modernise its approach to application delivery and new ways of working. Using tools such as AWS Lambda, API Gateway, Lambda@Edge and Route53 ensures a cost-effective solution that delivers a substantial improvement over the previous solution.

The BJSS solution uses the following AWS services:

AWS CloudFront Enables the Search application to make use of the HTTPS protocol for S3-based web content on a given domain name. The CloudFront Geo-Restriction feature restricts access to Search to requests originating from UK IP addresses.
AWS API Gateway Provides the endpoints used by both the Web-based and Native applications. The Gateway is configured as a proxy to pass the full request (including header) into the Lambda Function which encapsulates the functionality for the Search application.
AWS WAF Restricts access to the service to known IP addresses.
AWS Lambda The DVSA Search Lambda function is written in Java and encapsulates routing within the function hence the use of a Proxy configuration on the API Gateway for the application.
AWS Lambda@Edge Enables modification and addition of security response headers.
AWS RDS Contains an import generated from several data warehouse sources.
AWS KMS Supports encryption of Lambda Environment variables.
AWS CloudWatch Provides default and custom metrics from the application to DashBoarding and CloudWatch alarms.

By working collaboratively, DVSA and BJSS Technical Support Service have:

  • Increased users experience by deploying on a robust platform
  • Aligned with DVSA corporate direction of mobile, Cloud and server-less strategies
  • Ensured a seamless transition to Technical Support Service support
  • Provided knowledge transfer to DVSA Corporate Systems DevOps team thereby increasing their on going capability
  • Ensured DVSA Search is consistent with the best practices of other DVSA platforms supported by Technical Support Service, increasing efficiencies in supporting and maintaining them
  • Delivered a solution which has received wide acclaim within DVSA and has enabled frontline staff to undertake their operational duties more effectively through more easily accessible intelligence and vehicle related data

Carl Austin UK Chief Technology Officer

With Cloud quickly replacing the world’s data-centres and server rooms, the “journey to Cloud” is already well underway for many of us.  

Read the Thought Leadership

But what will Cloud look like over the coming years? After all, the rate of change within this space has been such that some applications that were delivered to, and developed for, the Cloud as recently as a few years ago could already be considered “legacy”. This relentless progress demonstrates how vital it is to have a view on the direction of change – otherwise new applications delivered for the Cloud may themselves become legacy in a few years’ time. We believe there are several key “Cloud futures” that will affect the ways in which the market will adopt Cloud platforms, in addition to designing and building software. BJSS’ Cloud experience is significant, with a diverse client-base that spans multiple industries and at many different stages of their journeys. This experience, combined with the views of our consultants and partners, has informed my views on the future of Cloud.

The death of private Cloud and the truth of hybrid

Death is a pretty final term, but the private Cloud is on a steep slope of decline. What is private Cloud (or “false Cloud” as Werner Vogels famously called it) anyway?  I don’t think the definition matters much more than the fact that it’s in your data-centre. That means you are putting significant time and money into the operational aspects of running your own Cloud. Not only does this often become significantly more expensive than public Cloud alternatives, it also diverts focus from generating direct business value (unless, of course, you are yourself a private Cloud business!).

You would also be missing out on a wide range of benefits of the public Cloud: speed of progression, truly elastic scaling and highly integrated ecosystem, full API driven automation, the list goes on. Many senior managers and IT departments will not recognise the value of these additional benefits, but the opportunity cost is significant – perhaps affecting bottom line at an even higher cost. This amounts to some pretty good reasons to go public instead of private.

From our experience at BJSS, most organisations see it that way too. The risks associated with public Cloud – security, skills, regulators, overspending – which are oft the sales pitch for private Cloud are over-sold and dissipating – with even those who have implemented private Clouds putting in place public Cloud strategies. Often, dissention in the ranks follows with splinter groups using their credit cards to create public Cloud accounts. These strategies often start with non-production systems only, but I’ve no doubt that they will end up replacing private Cloud for production systems too, potentially writing off significant investment. I would strongly recommend that organisations looking to invest in private Cloud take a step back and reconsider for a moment.

So, with private Cloud in the throes of death, what about hybrid? The concept of an IT estate that spans across the data-centre and the public Cloud might sound appealing. But, in reality, the nirvana of a homogenous environment is somewhat unattainable. In the long run, hybrid Cloud is simply a stepping stone into a public Cloud future. It is a means to an end rather than a sensible long-term strategy. From a strategic long-term investment perspective, the same arguments against private Cloud apply to hybrid. The public Cloud providers all have hybrid stories, but these aren’t based on a vision of the future where companies retain their data-centres, they are based on the need to address a proportion of the market who aren’t yet ready or able to let go and to ensure they win those clients’ business in preparation for a future when they are.

Who wants infrastructure anyway? Application delivery for the Cloud

Unless you are a hardware vendor, data-centre or Cloud provider, delivering infrastructure probably won’t generate direct business value. It’s more likely to be seen to be a necessary expenditure to support the delivery of software and services that do generate value. Infrastructure as a service looks to reduce the cost and complexity of infrastructure. Serverless, on the other hand, looks to release it. Of course, there are problems for which serverless solutions are highly relevant and others where they just don’t fit the bill. I fully expect the balance to shift over time, with more and more applications built upon a serverless backbone and a greater range of Cloud services delivered in this model. Our experience suggests that, not only is the infrastructure operation effort removed, but it is also possible “double-dip” by making significant savings to running costs. Who doesn’t like the sound of that? The progression of serverless also has other effects.  The ’you build it, you run it’ ethos will become easier to attain, while demand for specialist Cloud infrastructure and sysadmin type skills will decline. This will be counter-balanced by significant increases in demand for engineers and application architects experienced in serverless and Cloud native delivery patterns. This will fuel an increase in applications able to deliver change many times a day – so the demand for continuous deployment experience will accelerate.

Why pay for something that in likelihood you’ll never use?

Many organisations are planning to develop software to be run on more than one Cloud provider, whether as a form of Disaster Recovery, or to protect from vendor lock-in. In many cases this approach chooses to forgo the Cloud native features and offerings of individual Cloud providers in favour of commodity IaaS capabilities or layers of abstraction such as containers. This is a form of insurance, but the premium can be high (effort to deliver Cloud agnostic software can be considerably higher) and the risk proportionally low. Much like decisions to back out of private Cloud investment, I’ve seen organisations stepping back from previous decisions to remain agnostic of Cloud provider. This doesn’t mean that they don’t consider the potential impact of native service adoption – it means they have. I believe that this trend that will likely continue, impacting one of the use cases for containers – abstraction from Cloud provider. I would recommend carefully considering and revisiting reasoning for avoidance of lock-in and whether it genuinely makes business sense. Native capabilities change quickly. It’s easy to make the decision in isolation, perhaps as part of company Cloud strategy, without fully considering its effect on application delivery.

Security experts: change behaviours or get left behind

As the speed of software delivery is pushed faster, accelerated by Cloud native architectures and serverless capabilities, much of the security profession could be left behind. Practitioners need to adapt and change behaviours to avoid this. Security is often seen as a blocker to progress and the outcomes of this are both undesirable. Either a team finds ways to subvert security, which is likely to make the software less secure, or the team slows down its delivery. I believe that the answer is the adoption of DevSecOps. This approach advocates moving security forwards, making it the responsibility of the entire delivery team and utilising security expertise as an aid to that shared responsibility. It encourages use of automated security testing, red & blue team simulations, comprehensive security monitoring and threat intelligence, while discouraging the most common complaint of security: always saying no. The security expert becomes a mentor to teams. Success being that their input is required less frequently, and that they become a reviewer and validator day-to-day, allowing them to focus their efforts on truly specialist input. Those who don’t take this approach, who are seen to be blocking advance, will find themselves marginalised by the speed of progress.

I would encourage security professionals to embrace innovation within their profession, to develop new ways of working and to revisit and improve these currently. I would also encourage engineers to work threat modelling into their repertoire and consider how tooling can help to automate some aspects of security assurance within the build and deployment pipeline.

The core(s) of the AI revolution

AI is on track to be the next technological, potentially even social, revolution. Cloud will be at the core of this movement. Not only is the ephemeral nature of Cloud computing an excellent fit for machine learning, but Cloud providers are bringing AI to the masses with APIs that significantly reduce the specialist knowledge required to build intelligent software. This will be key to the future of Data Science, allowing access to advanced predictive and prescriptive analysis to those who typically sit outside of the Data Science profession, though still requiring some specialist knowledge. Perhaps, more interesting for the expert Data Scientist, is the specialist AI hardware that the Cloud will shortly make available to all. Google and Microsoft are both developing specialist machine learning processors, while AWS provides FGPA and GPU instances, having formed a close relationship with NVIDIA who is targeting the AI development market and continuously improving high level programing interfaces to its chipsets. These moves demonstrate the level of investment that the Cloud providers are putting into AI so they are at the forefront of the revolution. However, with this drive towards machine driven decision making, the reasoning behind decisions is obfuscated. This presents an opportunity to a hacker, a smokescreen for malicious interference that aids avoidance of detection, enabling subversion of the decision-making process to the benefit of the hacker. Effective monitoring and explainability of decisions is paramount to ensuring the integrity of these capabilities.

In conclusion

Cloud will continue to define how we design, build and deploy software applications. It will also play a central part in the AI revolution. Don’t fall into the private Cloud trap. Utilise public Cloud and benefit from the use of Cloud-native capabilities with an educated approach to vendor lock-in. Be forward looking in your approach to security and encourage an environment where everyone is both security conscious and jointly responsible. Above all, consider how the direction of change affects your business, applications and even your profession, ensuring that both you and your new software aren’t considered legacy in just a few years’ time.

“BJSS’ understanding of Cloud technology, and the benefits it can bring, were crucial to the successful delivery of our exciting new consumer banking project.” Rob Liddell, Projects Director, PCT

Read the Case Study

Payment Cloud Technologies (PCT) offers an innovative product that allows its clients to deliver banking services via the Cloud. The platform delivers rapid market entry for reliable and robust banking-grade financial services which support any digital portal, including iOS and Android. As PCT’s delivery partner, BJSS designed and built the product. BJSS was selected based on its award-winning BJSS Enterprise Agile® delivery approach, and its platform agnostic experience of Cloud services. A Microsoft Azure-based technology stack was chosen based on PCT’s specific requirements. The product includes several WCF-based microservices that implement business logic and provide access to third-party systems. Combined with a customer-facing website and over-the-counter APIs, these services are packaged in Microsoft’s Cloud Service product and benefit from load balancing along with auto-scaling and self-healing facilities. A collection of data storage technologies underpin the system. Microsoft SQL Azure stores account records while Microsoft’s CosmosDB stores high volume transactional data leveraging its low latency properties with guaranteed high-availability. A collection of Microsoft Storage Queues connect the microservices and third-party systems, enabling the system to smooth out peaks in load while remaining responsive. This multi-tenant Cloud-based solution is scalable, secure and flexible, allowing PCT to provide an offering that can grow with its clients needs.

Putting it all together.

BJSS supports the entire development lifecycle from Proofs of Concept, to development and support – and facilitates a seamless transformation that embraces Agile and DevOps. This end-to-end solution allows clients to adopt Cloud successfully, digitise rapidly, and get to market quickly.

Delivering Cloud-Powered Loyalty for the Co-op

Read the Co-op Case Study

Context Membership is core to the Co-op’s business model and significant investments were planned to make membership user-focused, innovative and compelling. Growth projections are ambitious with an extra 1 million new members expected to join within a year. Technology is integral to achieving these ambitions, requiring collaboration between product management and delivery teams and rapid deployment of working software. BJSS Solution The Co-op engaged BJSS as part of a dedicated digital transformation team operating independently of the existing IT function to drive significant improvements to both the online platform and ways of working. Delivery flexibility, adaptability and responsiveness were improved by combining aspects of the proven BJSS Enterprise Agile approach with other contemporary delivery methods to create a bespoke Agile delivery framework. A new AWS-based Membership platform was designed and delivered using this framework and several Java microservices. The new approach delivered shorter development cycles enabling the frequent, iterative delivery of key components and new features. Cloud Deployment The platform makes sophisticated and pragmatic use of AWS services. It is primarily hosted in AWS but interacts with a number of supporting on-premises services and other cloud service providers. Production – The platform comprisesseveral microservices running in Elastic Compute Cloud (EC2) using Relational Database Service (RDS) PostgeSQL for online data and Redis ElastiCache for caching. The services are configured in Autoscaling Groups and a mixture of Application Load Balancers and Elastic Load Balancers are used as appropriate. Simple Email Service (SES) is used for customer communications and internal system alerting. Networking – Route53 handles the internal and external DNS and the project makes sophisticated use Virtual Private Cloud features to handle public-facing services, third-party cloud services and connectivity to existing data centre applications. Security – Development, test and production environments are segregated into different accounts with Identity and Access Management (IAM) used for finer grain control and any cross-account permission requirements. Web Application Firewall (WAF) protects public-facing components. Key Management Service (KMS) and Certificate Manager are deployed for storing and rotating keys and certificates, CloudTrail for audit logging and Trusted Advisor as a reference check for AWS-recommended security considerations. Bastion VPCs with jump-off boxes is used to control and audit access to each environment. Monitoring – Elasticsearch Service is used to provide searchable access to application (CloudWatch and other sources), network (including VPC Flow logs) and audit (CloudTrail) logs. Software Development and Deployment – EC2 is used to host the various CI/CD pipelines, source control, knowledge bases and project tracking tools used by the development teams, with Elastic Beanstalk used for deployment. Storage – Simple Storage Service S3 is used for a variety of storage needs: software build artefacts, data processing, application storage, log storage etc. Public Cloud hosting, which was adopted based on BJSS’ Cloud expertise, allows the client to quickly ramp its service up Business Benefits

  • A significant jump in transactions, turnover and visit frequency is attributed to the transformation programme.
  • The effectiveness of the new Agile delivery framework is acknowledged internally and is being implemented across the organisation.
  • The Co-op welcomed over 700,000 new members in the first four months of transformation, and has exceeded 1m additional new members in 2017.
  • Since launch, over £70m has been paid in member rewards, and over £20m to local causes.