Infrastructure

for all those AI nerds there’s been some pretty interesting announcements from Google number one anthro pics Claude 3 is now generally available on vertex AI Gemini Pro 1.5 and Gemini 1.5 flash are also generally available we’re over 700,000 models on hugging phe so you can use any of the models on hugging face with vertex AI for those not familiar hugging face is kind of like a repository like git lab but for AI models so people taking off the shelf models or creating their own um modifying them and then uploading them to hug phase the next thing that’s super interesting is context cing so you can use context cing with Gemini Pro 1.5 and Gemini 1.5 flash models and this lets you past some of the tokens that you have uploaded so if you have uploaded um video and you want to ask multiple questions about it you don’t need to upload that video each time which is obviously going to be charged you can upload it once and ask multiple questions same thing if you have chat Bots with very long instructions um or you’ve got a large amount of documents and you’re asking different queries around document um the final use case I think was interesting is if you have a code repository and you’re looking to fix a lot of bugs upload it once C that context and then can do a lot of careers against it reducing both the cost and the latency to get those insights um if you need help with any of this feel free to reach out always happy to have a CH thank you

so you started using cloud and your costs keep growing and growing every month it seems to be more and more money than you’re spending on cloud and you’ve realised it’s time to take a look and cut those costs down to something that’s more sensible if you’re using Google Cloud they’ve got the fin ops Hub and the billing manager where you can go and see where these costs are broken down they’re often broken down by project so you can kind of see where some of the hot spots are and to reduce that the next thing you should start looking at is Devon test workloads and people pay me to come in and consult and say hey do you really need your development workloads running 24/7 when your developers are only working 9 to 5 it’s pretty logical get that turned off when it’s not in use even better get those running on spot instances these are substantially cheaper but when Google have low capacity they will take them away from you that will kill the developer workflow but it will ensure that your developers are writing code that can tolerate failures which is key to running anything on cloud the next thing you want to do is you want to enhance the visibility you’re getting into where you’re spending money now this is done with labels so every project or every resource that you have running should have a label on it with an owner and that owner should get an invoice not an invoice but a report at the end of each month showing how much money they’ve spent that will Empower your team to understand that they might be spending money that they don’t know about and have a look and see if they can reduce that by themselves this is really simple with Google creating labels putting them on everything and then exporting all the billing data into B crew so you can slice it dice it run reports and figure out where you need to focus your cost saving another few things that often get missed is Right sizing computer machines so being a computer engine you can individually change your memory and CPU to right size it to your workload now a lot of people do this as a one-time exercise and they kind of guess it they never come back and revisit it there’s tons of reports in Google where you can go through have a look at these things and then save yourself considerable money just by getting rid of unnecessary resources that your machines aren’t using if you donate anyone have a look at this feel free to reach out thank you

so I’m going to break down a few Concepts you might have been hearing when people are talking about AI the first one is retrieval augmented generation or rag it’s a bit of amouthful but it’s really simple if you ask an LL question so chat GPT or Google’s Gemini it’s going to respond based on what it’s being trained on which is the context of the entire internet but nothing specific to your business rag solves this problem by taking your business data uploading it into a database so that when you ask a question the question can retrieve data from the database based on your business and then formulate a response that’s grounded in that this reduces hallucinations or llms making up nonsense and make sure that it’s using data that is real from your business now the other concept we have is chunking if we’re taking documents and uploading them into the database we don’t want to upload entire documents cuz we’re not going to send entire documents to the llm very expensive so we chunk this you could chunk via paragraph but sometimes you need the paragraph surrounding that paragraph to get the full context or you can chunk via headings now different things are going to work for different businesses depending on how your data is structured by fining the way we chunk and store that in a database so that the LM can retrieve it and swapping out the llm model we can optimize for your business making sure that you get the best results possible for the best price possible whenever a new llm is released we can also test that very rapidly and see if that’s going to give you better results or a better price if you’re interested in learning more about this feel free to reach out happy to have a chat with anyone on these subjects thanks

Getting Started With Google Cloud Transcript getting started with Google Cloud can seem overwhelming at first as with any cloud there are a lot of services that you can use and each has configuration options that can get you into trouble when I worked at Google Cloud I helped some of the biggest brands in Australia set up their cloud environments and I’ll give you a few tips that I learnt from doing that the first thing you wanna do is enable some structure trading an organisation and then creating folders in the organisation to keep projects organised and allow to give groups of users permissions to do things to those projects for example putting all the development projects in a folder called development and giving developers access to those and then having all of the production projects in a production folder maybe without access for developers or for other groups of people once you have the folder set up you need to set up identity and access management so as I kind of touched on that’s creating a group putting developers in the group and then giving that group access to the folder that contains the projects that developers need to work on to do their jobs we may not wanna give them access to the production folder at all or maybe we only give them read only access this is a super simple example and we can nest folders and get much more complex with it and any environment that we’re talking about is gonna have more complexity than that this is a simple explanation now we wanna start talking about organisational policies we’ve got a group of developers that got access to their projects and development folder that we still don’t wanna do anything silly like putting a cloud storage bucket on the internet so that anyone can see what our files are even if those are development mocked data having a data breach is not gonna be good in the headlines there’s a ton of all policies and each one of them needs to be configured and this one example appear for the cloud storage bucket we may need an exception for the public facing internet to be on the internet once we have all this set up we kinda wanna make sure that we’re managing with code if a developer does request that a cloud storage bucket be put on the internet we wanna see who requested that and why and track those changes the logical step here is using infrastructure as code we use Terraform the same as the best practice at Google Cloud that Google had professional services used when I was there and we can do the same for your business in 5 days excluding any complex networking some people are spending much longer on this and it’s really not that complex if this sounds too complex do reach out we’ve done this when working at Google so we know the best practices and we know how to set you up securely so that your business can scale on Google Cloud thank you

Getting Started With Google Cloud Transcript getting started with Google Cloud can seem overwhelming at first as with any cloud there are a lot of services that you can use and each has configuration options that can get you into trouble when I worked at Google Cloud I helped some of the biggest brands in Australia set up their cloud environments and I’ll give you a few tips that I learnt from doing that the first thing you wanna do is enable some structure trading an organisation and then creating folders in the organisation to keep projects organised and allow to give groups of users permissions to do things to those projects for example putting all the development projects in a folder called development and giving developers access to those and then having all of the production projects in a production folder maybe without access for developers or for other groups of people once you have the folder set up you need to set up identity and access management so as I kind of touched on that’s creating a group putting developers in the group and then giving that group access to the folder that contains the projects that developers need to work on to do their jobs we may not wanna give them access to the production folder at all or maybe we only give them read only access this is a super simple example and we can nest folders and get much more complex with it and any environment that we’re talking about is gonna have more complexity than that this is a simple explanation now we wanna start talking about organisational policies we’ve got a group of developers that got access to their projects and development folder that we still don’t wanna do anything silly like putting a cloud storage bucket on the internet so that anyone can see what our files are even if those are development mocked data having a data breach is not gonna be good in the headlines there’s a ton of all policies and each one of them needs to be configured and this one example appear for the cloud storage bucket we may need an exception for the public facing internet to be on the internet once we have all this set up we kinda wanna make sure that we’re managing with code if a developer does request that a cloud storage bucket be put on the internet we wanna see who requested that and why and track those changes the logical step here is using infrastructure as code we use Terraform the same as the best practice at Google Cloud that Google had professional services used when I was there and we can do the same for your business in 5 days excluding any complex networking some people are spending much longer on this and it’s really not that complex if this sounds too complex do reach out we’ve done this when working at Google so we know the best practices and we know how to set you up securely so that your business can scale on Google Cloud thank you