Allowed R6 · Within policy
{
"slug": "transformer-attention-and-qkv",
"title": "Transformer Attention: Q, K and V",
"difficulty": "hard",
"tags": [
"ml",
"technical",
"role-ml"
],
"is_free": true,
"locked": false,
"frequency": 0.667,
"last_reported_at": "2026-09-12T20:48:54.370147Z",
"url": "https://www.mokabla.com/en/c/cyshield/transformer-attention-and-qkv",
"agent": {
"page": "https://www.mokabla.com/agent/en/c/cyshield/transformer-attention-and-qkv",
"question": "https://www.mokabla.com/agent/_call/get_question?company=cyshield&question=transformer-attention-and-qkv"
},
"company": {
"slug": "cyshield",
"name_en": "CyShield"
},
"body_md": "## What they ask\nThe first technical for an NLP or AI role is usually one team member and it opens on the architecture. Draw the transformer for me: what is in the encoder stack, what is in the decoder stack, and what is different between them. Then, block by block:\n\n- **Which attention is in which block.** Self attention in the encoder, masked (causal) self attention in the first decoder block, cross attention in the second. Say why the mask exists: at training time the whole target sequence is present, and without the mask position `t` would read position `t+1` and the model would learn nothing useful.\n- **What Query, Key and Value actually are.** Three learned projections of the same input. The query asks \"what am I looking for\", the keys advertise \"this is what I contain\", the dot product of the two gives the weights, and the values are what gets mixed. I will ask why we divide by the square root of `d_k` and what happens to the softmax if we do not.\n- **In cross attention, where does each one come from.** The decoder supplies the query from its own hidden state; the key and the value come from the encoder output. Candidates who have only read about self attention usually stall here.\n- **Multi head attention**: why several heads instead of one wide one, and what the concatenation plus output projection is for.\n- **Pretraining objectives.** Masked language modelling for encoder only models, next token prediction for decoder only, denoising or span corruption for encoder decoder. Then: which family would you pick for retrieval, for classification, for generation, and why.\n\n## What they look for\nThat you can move between the picture and the maths without hand waving. They are patient with a candidate who says \"this part I have only read about\" and hard on one who invents. Expect the interview to switch between Arabic and English mid sentence; answer in whichever you think faster in.\n",
"follow_ups_md": "- Why positional encodings at all, and what changed with rotary embeddings?\n- What is the complexity of attention in sequence length, and what do you do when the sequence is long?\n- Encoder only, decoder only or encoder decoder for a semantic search feature? Defend it.",
"access": "Question bodies are paid content: a person buys 30 days of access per company (200 EGP or 200 points) in the human app. Agents get titles, tags and difficulty; bodies only for free questions."
}