ai

To create an LLM, pretraining and finetuning is involved. Pretraining is done through self supervised learning on raw data to create a base or a foundation model. In self supervised learning, an LLM uses the labelled data generated by itself to train itself. Fine tuning is more specific to the use case. Instruction and classification are two types of fine tuning. InstructGPT is a fine tuned GPT-3 model using instruction finetuning. Transformer architecture is used at the core of an LLM which was originally used for machine translations. It has an encoder and decoder. Encoder takes the input text and converts it into numerical values. Decoder takes the numerical values and generates the output text. BERT is one of the variants of transformer architecture. It’s mainly used for masked word prediction, which means the input data can consist of what comes before or after the token to be predicted. GPT is used for next word prediction. Autoregressive models use previous outputs as the next input. Zero shot learning means the model doesn’t require any examples to convey the requirement and expectation of the task. Pretraining involves data preparation which includes converting the text to tokens and then to vector representations. Then a sampling strategy is used to generate input output pairs. The process of converting raw text to vector representation is called embeddings. LLMs are not compatible with categorical raw data. They need vectors for mathematical computations. Embedding models like text embedding models are used for this. Word2Vec is one of the landmark algorithms. Dimensionality of embeddings determines the expressive capacity of the model. The first step is tokenization. Take the text from the public domain and split the characters. Whether to include whitespaces is a business or technical choice. The tokens generated from this raw text are now ready to be converted into token ids. From these tokens we create a set of unique tokens and assign a numeric ID to each. When a new text is used as input, each token of that text is converted to these numeric IDs from our vocabulary.

database

Using explain and explain analyze is one of the ways of query profiling. Using these keywords in any query gives you a summation of algorithms used to execute that particular query. Why bother? Because when the tables grow in size, some of the queries start taking a lot of time and this latency gets added up in the API response time which is evident on the UI, to the user. Optimizations can be on different levels. You can use a caching db to avoid querying your master database, you can normalize your table to divide the fields into atomic level, you can use client side caching, you can avoid repeated calls to the db in the middlewares and controllers, you can improve the queries by avoiding fetching the columns that are not needed or avoiding sub queries etc. But right now I wish to discuss the impact of indexes on query optimizations. Let’s say you have done these aforementioned things and still want to reduce the latency further. Creating an index table can be helpful but like everything, excess indexing is also harmful because in order to reduce the time complexity you might increase the space complexity. Now, it depends on the individual case or a project or the budget. But let’s draw a line that we don't want to go overboard with indexing but use it definitely. But first, let’s define the playground. It has these columns and I have populated it with around 7 million rows.The table’s size is 1.6 GBs. Apart from the table’s size, what separates it from the real world scenario is its auto increment id as a primary key, that fact that i’m saving the address in the same table as events, and there can be a few more to point out which are escaping me at the moment but that is not important for now. It’s not a real world table but emulating the real world table will not contradict these points but only amplify them. Starting with a simple query which fetches all the events happening in the current month. It’s using EXPLAIN ANALYZE which will run the query and give us behind the scenes as well. It’s fetching all the columns and has a simple where clause which is checking only one column which is “date”.

ui/ux

In my example I have a total of 50 items in the list and here is the comparison of the number of total html elements in the DOM. Without virtualization, it has 197 nodes. With virtualization it has 27 nodes. This is an 86% reduction in the DOM nodes. This factor can change based on the elements each list item has. Virtualis. helps in reducing the number of DOM nodes which reduces the heap size occupied by the references of these elements. It maintains the frame rate while scrolling in case of longer lists. Let’s take a look at how this is achieved. List Virtualization needs the data that is to be shown as a list, let’s say n in the length. Then we decide how many list items are to be shown at a time, let’s say t. Based on this we will control the range of items that is being currently shown to the user. As the user scrolls down, we increment the window in a way that it starts showing the next set of items. So, if it starts with index [0, t], upon scrolling it will change to [1, t+1] and so on. To decide when this increment should happen, we use the scroll height of the container element. Let’s go further in detail. Each item is given a fixed height in this list and their position is set relative to the container based on their index and height. If the fixed height of each item is 10px, then the first element at 0, second at 10px, third at 20px…As the user scrolls, dividing the scrollTop with the default item height, we get how many items down has the user scrolled. This value is where our window starts and adding t in this value gives us the last element of our window. Together, this dynamic change of window range and the respective position of each item in this window gives the illusion that the user is scrolling through a static list of items. To understand this whole workflow better, break it intentionally or introduce some bugs. First, break the part which is updating the range of the list to control the visible elements. Then, break the part where we are setting the positions of the elements based on their index. Then, you will understand the significance of each logical component involved.

ui/ux

Let’s start with image optimizations. Nextjs has its own Image component that manages the loading and rendering of the image on its own. Even if it is used in a server component, this image component can be lazily loaded and rendered on the client side. But this does not cause any cumulative layout shifts. It takes care of that. That layout shift is prevented using the width and height you provide as props to this component. You can see that in this blog as well if you throttle the network settings. For src, if it's an image hosted at any remote bucket then you should use remotePatterns in nextjs config file to avoid the external domain error. For relative paths, you can use localPatterns. This is not an optimization but just a code pattern. blurDataUrl expects a base64 url string of an image that is less than 10 by 10 pixels and when clubbed with blur placeholder, it shows a blurred placeholder image while the actual src image is being painted on the screen. If you want to use a custom component as a loader, you can even use that and specify the loader key of the image in nextjs config file. If we go into the nextjs config properties for image, there are a lot of things that we can control like size of images for different viewports, formats of image allowed. Then comes the concept of prefetching. It’s achieved through the Link component. This component is an alternative to html anchor tag. If your page has some in app links/routes, you can choose to prefetch their respective js beforehand. It takes a boolean value for prefetch prop. If true, as the page that uses it is painted, it will fetch all the compiled js of the page that this link is referring to. You will see the difference in the load times. These calls were made while being on the landing page and not clicking any of those links. You can only check this on your deployed link, your local dev server won’t be showing you these calls even if you specify true as its value. Notice how it says _rsc which stands for react server component. Since all these pages are static pages, the whole RSC payload of that route is fetched.In essence you are just deciding whether you can afford to keep the latency added by these calls in the route navigation. If not, you can prefetch them. But this is a case of a simple static page. When there is a page which has server rendered components using api calls and some client components, just relying on prefetching does not reduce the responsiveness. One reason being, for dynamic routes, the whole RSC payload is not prefetched. Another reason being, even if it is, provided you are using the loading file, due to the large number of components and libraries imports used in it, its js chunk gets relatively bigger and there you have to make use of code splitting.