Real AWS DevSecOps Engineer Interview | Scenario Based | DevOps | Experienced | Docker | Kubernetes

[Music] So I have your resume here. Um, so we can go through this and you can brief me a bit about your experience, what your core skills are, and we can take it from there.

Yeah. So let me give you a brief intro about myself. So my name is Dashas Singh, having close to 9 years of experience till date.

There is a bit of background noise that I hear. Is it? Uh, I am at, yeah, someone's place and everybody is working from home.

Uh, okay then allow, yeah, allow me 2 minutes. Let me go to another place.

Okay. Yeah. So this is for the senior DevOps engineer role.

Okay. Yeah.

Okay. Am I visible and audible right now?

Yes. Yeah. So yeah. So basically, I'm having total close to 9 years of experience as of now till date in IT industry where I started my career as a software engineer trainee working on PHP technology mostly developing websites using PHP as the back end language and yeah going forward I upgraded myself or added front-end skin like HTML5, CSS3, JavaScript, jQuery, Angular, React. So basically I was a full stack developer initially for the initial phase that is three years and after that I upskilled myself into cloud computing and coming to cloud I got opportunity to undergo training and certification for AWS, Azure, and IBM Bluemix but yeah got an opportunity only to work on AWS platform as an application engineer or cloud engineer mostly engaged in the migration activities and during the same during the cloud journey I got an opportunity for a project where the DevOps practice was to be integrated or implemented from there starts my DevOps journey and till that I I'm working as a core DevOps engineer coming to DevOps initially started with the tool stack or stack as sort big bucket Jenkins gitlab and docker kubernetes started with self scripting but later moved to python scripting and since last two plus years as uh I been I've had been associated with banks so I got exposure to the security part also so hence as of now I'm working as a dev sec ops engineer you can say okay so yeah I'm coming to current profile currently working for HDFC bank as dev sec ops engineer and the roles and responsibilities include mostly pretty much similar to a DevOps engineer apart from you're considering the security practice over our devops platform.

So okay so one one quick question um I got a good good idea about your profile how about this sort of transition like from Standard Chartered to HDFC?

Um so usually the Indian banks they don't have that digital presence or you know you can say not so well for fast-paced kind of banks.

So and and and also the salary part like was HDFC able to meet your salary and all other stuff?

Uh yeah see this yeah this is another story when I opted or when I decided to move from SCB. So I had the multiple offers like um not from India, I had offers from abroad also and I had planned or the plan for which I moved out of SCB was just to go abroad but at the last stage you know due to family issues constant and all I could not yeah make it although I had everything with me the visa also started but I have to end it and at that point of time I got offer from HDFC and the tech stack uh that you are talking about Indian banks I was well aware of Indian banks don't have the tech stack but in HDFC these people they just had started two new verticals that is digital factory and enterprise factory on July 2021 it had been around 8 or 10 months when I was interviewed after the formation of their two verticals that extract the need skills were very good and the people or the leaders who were hired to drive this they basically come from mostly consulting and startup background. So thinking that or assuming that that I will not have constant tech or anything like in banks we used to have much restrictions and the legacy technologies. So thinking that only I joined HDFC and yeah the decision was correct and the salary part was taken care.

Yeah. Yeah. Salary they gave good salary like good okay cool all right so we will get into the discussion so brief like basic questions what's your understanding of DevOps, DevSecOps maybe DevSecOps if you can explain we can take it from there.

Uh okay DevSecOps is not a technology or it's not a tech stack rather it's a practice which is implemented over multiple organizations and coming to multiple organizations it is not specific to any category maybe fintech maybe any bank or maybe e-commerce but yeah that SecOps should be imposed or should be enforced to each and every organization irrespective of the domain.

There are multiple benefits out of integration of DevSecOps which leads to a formation of a security layer over your organization structure. The second one having a security gate where chances of possibilities are there where the major part of any organization is the data is compromised by various means. So yeah that can be prevented by DevSecOps. Third one is performance agility and obviously the last one again the security of your whole infrastructure.

So, so what do you mean by SecOps in this whole phenomenon and how do you accomplish that?

Okay. SecOps significance uh the bridge between your security and operations. Ideally the bridge between the developer and operation is being filled by DevOps and security coming to add I can say to add cream on the cake. So that way I can define.

Okay. So any specific tools that you have used to enable the security?

Yeah. So with God's grace I can say I got an opportunity with the leader with whom I am working as of now. He gave me a free hand in driving a tool named Checkmarx which is used to scan the vulnerabilities. So from HDFC point of view for our vertical I am the single point of contact who is yeah collaboratively working since last year with the vendor in driving this Checkmarx. It started all with the on-premise and now we have moved to a cloud SaaS model and the scan types include your SAST, your SCA and now we have included KICS also which secures your infrastructure maybe docker file maybe Kubernetes or Terraform anything or your manifest files. So apart from that we are using SonarQube and for SonarQube also I may not say the single point of contact but I am the single point of failure for SonarQube but for Checkmarx I'm single point of contact as of now we haven't any point of failure yet yeah those two yeah two tools we are using as of now.

So did you evaluate Checkmarx comparing with some other product? How was this decision made?

No. When I joined just two months ago the decision was made that Checkmarx would be onboarded. Already it was being onboarded to bank but to our vertical it was me who onboarded.

Ideally our yeah no ideally our responsibilities was that we as a DevSecOps engineer we should only help to integrate those tools in our pipeline. That was the major responsibility.

But as it was an open playground and there was no restriction and I took adult responsibilities and I drove it or I drove it.

Okay. Any specific benefits that have come through any you can quantify using Checkmarx?

See basically yeah these are all parameters would have come into picture before making the decision of onboarding into bank but coming to your question what has been the benefit of using this as user point of view I can say it's pretty I can say straightforward and pretty flexibility for any users they can the integration was not although the integration was yeah little complex or had a twist but yeah consuming those from a user point of view it's pretty yeah straightforward and they can generate the report which again based on the approval can take ahead with their release process or with the further process of the but what what is the benefit you have got after integrating this tool?

Okay, the benefit is that we we have a ground level up check as of...

operation is being filled by DevOps and and security coming uh to add I can say to add cream on the cake. So that way I can define. Okay. So any any specific tools uh that you have used to uh enable the security?

Yeah. So with God's grace I can say I got an opportunity with the leader with whom I am working as of now. He gave me a free hand in driving a tool name as check marks which is used to scan the vulnerabilities. So

from HDFC point of view for our vertical I am the single point of contact who is yeah collaborative working since last year with the vendor in driving this check marks. It started all with the onremise and now we have moved to a

cloud SAS model and um the scan types include your SAS your SCA and now we have included kicks also which u secures your infrastructure maybe docker file maybe kubernetes or terapform anything mhm or your manifest files. So apart from that we are using

sonar cube and for sonar cube also um I am I I may not say the single point of contact but I am the single point of failure for sonar cube but for check marks I'm single point of contact as of

now we haven't any point of failure yet yeah those two yeah two tools we are using as of now so did you evaluate check marks uh comparing with some other product How was this decision made? No.

Uh when I joined just two months ago the decision was made that check marks would be onboarded. Already it was being onboarded to bank but to our vertical it was me who who onboarded.

Ideally our yeah no ideally our responsibilities was that we as a dev secops engineer we should only help to integrate those tool in our pipeline. That was the major responsibility.

But as it was an open playground and uh there was no restriction and I took uh adult responsibilities and and drived it or I drove it.

Okay. Any specific uh benefits that have come through any you can quantify using check marks?

uh see basically yeah these are all parameters would have come into picture before making the decision of onboarding into bank but uh coming to your question what has been the benefit of using this as user point of view I can say it's

pretty I can say straightforward and pretty flexibility for any users they can the integration was Not u although the although the integration was yeah little complex or had a twist but yeah consuming those from a user point of view it's pretty

yeah straightforward and they can generate the report yeah which again based on the approval can take ahead with their release process of with the further process of the but what what is the benefit you have got after integrating this tool? Okay, the benefit

is that we we have a ground level up check as of now because we we haven't integrated over the pipeline. Apart from that we have all also integrated in the ID plugins of the developer assume me as a

developer who is developing uh any app or any website with XY Z text tag and sorry are you saying you have not integrated in the pipeline yet? No pipeline also integrated and we have also in the ID also. Okay. Got it. So

yeah. So there are two level of yeah sanitization done. First is from the ID level like not in the bank environment or or not in the in the dev secops environment rather in the developer environment and when they are pushing to

our environment then our pipeline takes care because in our pipeline it's already integrated. What are the benefits you have received so far with this tool? Uh benefits we have received we have uh the vulnerable free of uh applications. So so that's the primary

thing of getting what kind of issues has this reported? Do you remember any specific or any any type of issues that it reports? uh see uh see there are multiple scan categories uh under check marks uh starting from

sash which is source code application then hcs source composition analysis and gigs so multiple features multiple vulnerabilities comes under different categories for SAS it is pre-built like until unless your application is built you can have a scanning under SAS category for coming

into HCA when your application is built you need few dependencies libraries or few third party libraries which helps in building your application right so that comes under source code uh mean analysis so okay source composition analysis third one is the kicks which points to a

repository in so in our bank we have a single pointed repository or a standardized repository where our helmcharts docker file resides. So that is taken care by kicks.

So the main advantage of this is that we are we are sanitizing the application completely. We are scanning the vulnerabilities. We are fixing the vulnerabilities at a very granular stage.

Give me one or two example of these vulnerabilities. Um see it depends again. Uh okay, I can say as of now recently I I got one uh from a specific project.

Um it was from a docker file while scanning the docker file via kicks. So it had multiple or three multiple errors. The first being um those people of those project team they haven't met the standard of of creating the docker file like in bank we

have created a standard of creating a docker file. So there are numerous point yeah starting from the size then using uh a standard or basic image. So these are all the few points which we have set as the best practices of creating a

docker file or a docker image. So so these people they haven't met such that's very basic right standard not followed is uh even looking at with your eyes or just a code review can can get those kind of issues. Okay. But then again

it's a manual approach right? Yeah. Yeah. What is that SC? What is the tool?

Check marks has reported any such vulnerabilities. Typically we get so many issues, right? You must be scanning uh your mobile applications or your web applications, docker file, any two or three like what happens in in our environment. If we get some specific

issue, we get like 10 or 20 or 50 times that issue. Like this issue is at line 50 in this file, then another file. What are those top anything that comes to your mind that this tool helps with? Yeah. So previously uh I got uh

one report uh yeah going through one report uh it was uh yeah one developer they have uh drafted the credentials they have drafted the credential I guess I could not remember it was I guess DB name or some

credentials I I'm not sure for which exactly but they have Yeah, I integrated the username and password including the token number. So that was highlighted on the second or third point again very spec very like even a basic

code review would have revealed that again not something yes tool helps but no no you exactly I do understand your point something that with the naked eyes you can't easily detect or uh a tool would probably be better fit for that or

leave the tool what are the top security issues In general Hindi there are 10 categories or these guys follows see inside check marks they have uh I could not pronounce it O W so that can uh yeah the check marks have been

integrated or they have so if you standard if you say follow this standard they would bring those rules and the rules are already predefined for or means organization right. So when the scan initiate uh it makes sure that the vulnerabilities

which are being thrown out or the scanned result are being aligned with those standard or policies right if not then obviously it will it will be reported okay so what are those policies any any idea what what those I haven't

by hated it to be honest okay so what happens when you receive an issue your pipeline reports or uh what's the workflow Okay. So before conveying the workflow, yeah, let me give you a short background. We have a team here

like a governance team that is ISG. From our dev secops point of view, our work ends on integrating the pipeline and generating the report. Post that the developer team needs to take this report and have a clearance from

the ISG. Okay. So this is the workflow. And coming to uh the technology end we have integrated not only to only initiate the scan in the pipeline rather we have me or customize the logic in such a way that once the pipeline finishes we

have yeah set two more ad hoc enhancement which I just had released last week only. uh like we have set the quality threshold get for sash and HCA as of now kicks is not yet being integrated uh or being released for sash

and hca I have set the quality threshold go yeah based on the discussion so assume I have set five or 10 number of vulnerabilities to be passed for my dev environment or for my prod environment so if it breaches 10 or five the

pipeline will abort or the stage will abort and It will about the pipeline also. Got it.

But yeah, but the report will be generated and it will be shared to specific number of email. Yeah. To specific number of email users which are being drafted or which have been part of our genkins. NB5. We we have

completely parameterized and here we follow the shared lip structure. We have integrated this over our pipeline. Okay.

by creating of shared label so that as a user if you consume you can directly invoke the function on your genkins and you are good to consume our scan.

Okay. So give me like some more details as to um anything around highly available applications. How do you ensure one application is highly available and yeah some something around that uh well that was yeah that that doesn't comes in our bucket here in current

organization the reason being it's a very big organization and we have a separate clouds of cloud ops vertical for it okay so we are a centralized dev sec ops team okay who's whose mostly responsible or primary responsibility is

to create frame framework or platform that can be consumed across the bank in order to integrate their application into dev sec. So if some some application has to be architected your team will not be involved in from intra

point of view also. No no no as of now. No your profile says you have used terapform and civil so does that fall into your responsibilities?

uh currently I have used recently terraform but we haven't architected it rather we have created few modules for one of the project at at a very initial stage like last year only I did but after that the project which are coming

to onboard themselves into dex ops they have their own infra guy who have created the modules but we make sure that they follow the standardization which we have set like we have our own repository structure and they need to

push their code in that repos like mostly in so banks till date they have outsourced everything in the tech part and now we are building a in-house community are you certified you said you are certified on few stuff

uh past I did for AWS developer associate as of now planning for cubernetes the CK mostly by next month I will be doing that okay all right uh do you understand um the cloud well architected framework.

Uh yeah in past organization I I got exposure to this. So can you explain like what what is what do you mean by that?

Um cloud architecture it depends like u like can you be more specific into which type of design or like there is a standard term industry term actually well architected framework. So what are those pillars of well architecture uh that you usually follow and

see mostly what I have came across is three tier application structure application network layer and database layer so three tier architecture what I have heard and what I have came across till date in my developer and both devops j okay okay can you explain what

is three- tier architecture uh three tier architecture. It combines your application layer, your networking layer and your database layer. Your application layer comprises of your all source code which are required to build your application and your network layer of signifies the piece of a part

or cake which allows you to have a connection between I can say a source and destination or between two layers. And coming to your database layer which is most significant. It comprises of all your I can say the exact keyword for is

this all your sensitive information all your sensitive data okay to be stored at a place you can turn that as a database.

What what what three architecture layer these are for application for infrastructure. Sorry. So you said this this is a three tier architecture. This is for for an application.

Yeah, for any application or uh I can say any new project if it comes up first we ask what is the so your your code code is uh one layer followed by networking and followed by secrets.

um like um I can say this three should be the coming under the category of application. Yeah, there is one more layer which is called user interactive layer or so that makes it four tier architecture.

No, it's a three tier. First is the user interactive layer. Then the second one is the application layer which comprises of uh this networking and database also inside that as a subcategory. And uh the last is I guess the network layer. I I

could not recall as of now but yeah this two I can recall that is the user interactive layer and the application layer. Okay. Do you know what is public subnet private subnets? Yeah in cloud app what is the difference between these

two? Yeah. Major difference. Not that this is public, this is private. No. Uh see the thing is that uh uh the one thing which are distinguish or make a differentiate is uh from a public subnet is used which can be accessed globally.

Okay. Without any restrictions. Okay. That is the basic different approach. Yeah. uh but coming from a tech point of view if I want to store my sensitive information definitely I'll not go find it with the public because that will be

compromised like my data will be compromised right or that is that is not the difference what is the key difference that is the reason to have uh public and private subnets I guess this is the basic the main reason why you should

have public yeah that is correct main reasons to have but what is the key difference between a public subnet and a private subnet?

Um as as for my experience anything from routes or networking point of view in public it is two-way accessible but in private it is means one way like a unidirectional and birectional I can say.

Okay. All right. Uh have you performed any disaster recovery in your environment by choice? To be honest, current did it.

In past also I could not because past was also a bank. So we did only P but I don't know that was integrated or not because by that time I left over. You are you are mainly on AWS right?

Uh currently we are not any cloud specific. Okay. We have here AWS and GCP including Azure. So we are not any cloud specific. Okay. Have you ever engaged into cost optimization activity?

Last quarter I was engaged on a cost optimization but it was related only to genkins live agent not for any project or not for anything this thing but it was a small P where uh the genkins agent which we is being hosted or which we are

using as of now for our vertical how do how can we cost optimize it by various considering various parameters that was a P also. So what what was your recommendation?

Uh our recommendation was because currently it was on premise so we have to give a strong just a justification uh to opt for a GCP one. So, so those were research and yeah those one tested with proper justification like

for onremise we the developers even we when we try to execute job on our genen platform we face a lot of issues like for the basic issue the node not available execute is not available and and in most cases it takes a time delay

to execute the pipeline although we have optimized everything on our genkins file but still it takes a lot time. So those are few things uh a few parameters based on which we went ahead with the cost optimization.

Any any difference uh between these two two or three different types of load balancers uh AWS offers or Azure?

Uh AWS I can say what I remember classic application load balancer and network load balancer. Mhm.

Mostly we I have came across application load balancer. What is the difference between these three?

Uh see application load balancer I have mostly used or I have been uh in touch with uh in my past uh experience. So application load balancers comes into picture or has it benefits of a a pathbased routing. So

previously before coming into the bank domain I was part of a startup it was mostly a e-commerce platform. So we had multiple modules of microservices for our website. So there we have we were using uh so that uh we can configure or we can

route based on the module part which in result enhances uh the traveling handling I can say. So that's uh the best part or that's the benefit of using LB increases the performance and handles the traffic based on the user request.

But NLB also supports pathbased that's not the major difference between these two. Um as of now I can recall this and I can okay in your u environment how are you managing the secrets?

As of now we are handling secret using vault or in genkins we are creating the genkins key store but we are not the support or we are not the right uh team or yeah we are not the right team who

are taking the ownership of the genkins tool rather we have a dev support main dev support it's a vertical who takes care of uh the ownership of these tools like g and genkins also we only consume their services. So we recommend to have

a key store to store the credentials over genin and for this Kubernetes we use secrets. Have you designed any solution system design?

Uh system design I can say not individually influent but rather part of the system design. Anything you recall new anything you have you have been part of which was designed recently?

recently not but in past yeah during the initial phase of my current organization and on my previous organization previous organization on the mid uh we I was part of the design end to end uh it was a migration project from onremise to cloud

with integration of yeah dev sec ops so there I was what was the approach for the migration and what was your role in that uh my role was from DevOps point of view because we had already a cloud

architecture there who gave uh or who transform the on-remise architecture the legacy architecture into cloud with the various uh AWS services. So we as a devops point of view we consume their uh endpoints in our services and allow the

end user to consume our dev support platform. So me as a DevOps engineer, we automated or we created a framework which will uh pick or which will consume or fetch the application from their source code repository scanning those creating a build dockerizing it

pushing into for docker image scanning post that deploying into a specific name space. This was a end to end framework created by me and my two team members in my past organization.

Were you involved in any um discovery of resources on prem or was it just automation? I uh no I was just in the part of automation. Okay.

All right. Uh I think that is pretty much I wanted to discuss. Uh do you have any questions? Uh yeah I have few if you allow may I? Yeah yeah yeah please. Yeah. Yeah sure. Thanks. Uh so just wanted to check as I am already

being conveyed by the person who have scheduled this interview or have asked this uh the platform for us to connect.

So just wanted to ask about uh what exactly because uh as I know that capital one is also in bank. So what all tech stack like how can I differentiate between uh what text stack or what environment I'm

working as of now and what environment is there set up for your organization with respect to tools technology everything. So I think we have tried giving as much details possible on job description as well. We are uh also

multicloud. Um what other text stack? same Docker containerization, Kubernetes, [Music] um, Terraform and Cible pretty much that you probably would have been working on or or as a DevOps engineer have some understanding. Um, so text is play test.

um we try to look at senior engineers, senior um DevOps engineers to provide feedback and uh make changes to our text tag. So we need we need that kind of um profile as well.

Okay. So what was your other question? Yeah. Uh regarding um the uh work location or like in Bangalore, do you have a specific to any state or No, nothing like that. We are pretty uh open. Uh we have people uh coming working from

remote locations as well. So we are so people who are in Bangalore u we have office and they do join office on u weekly or any such uh major um occasion when it's required but as of now we are into that

mixed mode if people are yeah yeah so if they are remote they want to work from home they are we are flexible on So you provide remote work options also.

Yes. At least for now. At least it's open right now there is no restriction.

Okay. So it's it's not permanent but yeah as of now based on situation. Yeah.

Correct. Correct. Okay. Because it's a bank and and Bangalore may know where is the location of like your place your office. Capital one has u different offices. Mhm.

Um we have um offices in electronic city and um in near the airport as well.

You have one near the airport and and the second one electronic city. Electronic city. Okay.

Perfect. Uh uh yeah one last question with respect to the security tools. Uh will it be fine if you can uh uh share uh which tech which exactly security tools been used or consumed by your DevOps platform? Yeah. Yeah. We use multiple uh

tools not restricted to any specific tool. That's why most of my questions were not about tool as such. They all have what we have found is they share similar sort of reports.

So what was also was same but as it was onboarded and and you know working in bank environment if you work with senior stakeholder people who are quite senior and experienced and they have a long junior in bank they don't go directly

for the change or they don't adopt the change at a fast phase as we do in IT organization. So that's the reason that the major reason or that's a fact we haven't went to other tool rather we have done p for 45 also and for one

other tool it was white footing or something I forgot those two were done and we took it ahead with the senior stakeholders but that didn't worked out because it has already onboarded and the governance team or the finance team they

they directly counter directly they rejected that uh This tool is already we have onboarded and if we are going to onboard a new one it will take at least six to eight months with multiple approval. I think check marks has advantage that um it's

it has an self-hosted version as well. Initially it was a self hosted or it was hosted in a blank version and now recently we have moved to CX1 checkbox one which is a SAS model. We yeah they have hosted in a single EC2 instance of

AWS unpointed to or a single tenant which is dedicated to SDFC. Right. Right. Right. Correct.

So it that's not a restriction over here. We try to you look at capital one history uh we have been developing in-house uh softwares and making them open source.

Not sure you look at hyia dashboard that was developed by capital one and um outsourced made it u open source that's a dashboard that connects to different dev sec ops tools and give you a very holistic uh view of the

environment that's a open source project now so yeah so we have bunch of in-house tools and also um multiple tools depending upon which area we are uh connected within the bank.

Okay, that's that's cool. Then then you don't have much restrictions like time like you have restrictions but uh you have transformed them into a healthy culture or a healthy environment where one can get an open environment to correct not really open

by their choice but by the type of application they are working type of application. Correct. Correct. Got it.

Got it. Here also we are following same and we are hired or my managers or yeah my yeah bosses they are hired for sale but uh a thing which I learned in one year here in current organization things

don't change at as you want especially in bank and if it is a big organization it takes time and things move very slowly but but again we are trying to set up something which we may see after couple of Yes.

Yeah. That's it pretty from my all right. I'll close the interview for now and then uh I'll share the feedback uh with the HR. Sure. Sure. Thank you. Nice talking to you. Thanks. Yeah. Bye.

[Music]