Epsilon-Greedy Multi-Armed Bandit

A multi-armed bandit is a row of slot machines (arms) with unknown payouts. Epsilon-greedy balances exploring and exploiting: with probability epsilon it pulls a random arm, otherwise it pulls the arm with the highest estimated value.

To make the run testable, the random numbers are given. Start with an estimate Q = 0 and a count N = 0 for every arm. At each step t:

If random_values[t] < epsilon, explore: arm = random_arms[t]. Otherwise exploit: arm = the arm with the highest Q (the lowest index if several tie).
Receive reward = rewards[t][arm].
N[arm] ← N[arm] + 1 and Q[arm] ← Q[arm] + (reward - Q[arm]) / N[arm], the running average.

Implement epsilon_greedy(rewards, epsilon, random_values, random_arms).

Input

rewards: One row per step with the reward each arm would pay at that step.

epsilon: The exploration probability.

random_values: One random number in [0, 1) per step.

random_arms: The arm to pull at each step if exploring.

Output

Return the final Q estimates, one per arm.

Constraints

1 ≤ steps ≤ 10,000, 1 ≤ arms ≤ 100

0 ≤ epsilon ≤ 1, -100 ≤ each reward ≤ 100

Examples

Example 1

Input:
rewards = [
    [1.0, 3.0],
    [1.0, 3.0],
    [1.0, 3.0],
    [1.0, 3.0]
]
epsilon = 0.1
random_values = [0.5, 0.05, 0.7, 0.9]
random_arms = [0, 1, 0, 0]

Expected output:
[1.0, 3.0]

Explanation

Step 0 exploits; both Q are 0, so arm 0 is pulled and Q = [1, 0]. Step 1 explores (0.05 < 0.1) and tries arm 1: Q = [1, 3]. Steps 2 and 3 exploit the better arm 1. Result: [1, 3].

Example 2

Input:
rewards = [
    [2.0, 5.0],
    [4.0, 5.0],
    [0.0, 5.0]
]
epsilon = 0.0
random_values = [0.5, 0.5, 0.5]
random_arms = [1, 1, 1]

Expected output:
[2.0, 0.0]

Explanation

With epsilon = 0 the agent never explores. It keeps pulling arm 0, whose average goes 2 → 3 → 2, and never finds out arm 1 pays 5. Result: [2, 0].

References

Run the sample tests to check your code without submitting.

Example 1 Not run

Input:

rewards = [
    [1.0, 3.0],
    [1.0, 3.0],
    [1.0, 3.0],
    [1.0, 3.0]
]
epsilon = 0.1
random_values = [0.5, 0.05, 0.7, 0.9]
random_arms = [0, 1, 0, 0]

Expected output:

[1.0, 3.0]

Your output

Example 2 Not run

Input:

rewards = [
    [2.0, 5.0],
    [4.0, 5.0],
    [0.0, 5.0]
]
epsilon = 0.0
random_values = [0.5, 0.5, 0.5]
random_arms = [1, 1, 1]

Expected output:

[2.0, 0.0]

Your output

4 hidden tests also run when you submit. If your code fails one, you'll see that test. Solve the challenge to see them all.

Run Swift Online

Write, run, and debug Swift code online using a command-line Swift environment. Manage multiple files, save your workspace, and share code without installing Xcode.

Version

Pre-installed Libs

🚀 19,043 total executions (611 this month)

Output

Practice Swift with coding challenges

Solve hands-on problems right in this editor: run the sample tests, submit, and get an AI code review.

All Swift challenges

Easy

Rectangle Perimeter

Compute the perimeter of a rectangle from its sides.

You'll learn ArithmeticReturn values

BeginnerLearning-pathVariables

View solutions (12)

Easy

Distance Between Points

Compute the straight-line distance between two points.

You'll learn Pythagorean theoremSquare roots

BeginnerFunctionsLearning-pathmath

View solutions (7)

Easy

Days in a Month

Return how many days a month has, including leap years.

You'll learn Calendar rulesHelper functions

BeginnerConditionsFunctionsLearning-path

View solutions (6)

Easy

Average of Three

Average three whole numbers.

You'll learn ArithmeticDivision

BeginnerLearning-pathVariables

View solutions (6)

Easy

Dot Product of Two Vectors

Multiply two vectors element by element and add the results: the core operation of every neuron.

You'll learn VectorsDot product

Neural networksaimachine learning

View solutions (4)

Easy

Largest of Three

Pick the largest of three numbers without a built-in max.

You'll learn ComparisonsCombining conditions

BeginnerConditionsLearning-path

View solutions (4)

Easy

Prime Number Check

Decide whether a number is prime.

You'll learn DivisibilityLoop optimization

BeginnerLoopsNumbersStudent-beginner

View solutions (3)

Easy

Number Triangle

Build a triangle of increasing numbers.

You'll learn Growing rowsCounters across rows

BeginnerLearning-pathNested-loopsPatterns

View solutions (3)

Easy

Armstrong Number

Check whether a number equals the sum of its digits raised to the digit count.

You'll learn Digit extractionPowers

BeginnerNumbersStudent-beginner

View solutions (3)

Easy

Row Sums

Add up every row of a grid.

You'll learn 2D listsNested loops

BeginnerGridsLearning-path

View solutions (3)

Easy

Min and Max Together

Find the smallest and largest value in one pass.

You'll learn Single passMultiple return values

ArraysBeginnerFunctionsLearning-path

View solutions (3)

Easy

Count Pairs With a Sum

Count pairs of positions whose values add up to a target.

You'll learn Pairs of indexesAvoiding double counts

ArraysBeginnerLearning-pathNested-loops

View solutions (3)

🍎 About This Swift Online Executor

The CodeUtility Swift Executor lets you write and run Swift code directly in your browser - no Xcode, SDK, or macOS setup required. It runs real Swift compilers in isolated sandboxes to deliver accurate, native Swift behavior.

This tool supports multiple Swift versions - 5.7, 5.9, and the latest release - giving you the flexibility to explore new language features and maintain compatibility with older Swift codebases.

It’s perfect for developers learning Swift syntax, testing algorithms, or experimenting with code for iOS, macOS, and server-side Swift without needing to install local tools or simulators.

Each snippet runs securely in a containerized Swift environment, ensuring real compiler feedback and safe code execution.

More than a code runner

Use a multi-file editor, an isolated execution terminal, runtime-specific libraries, persistent workspaces, sharing, and real-time collaboration from the same executor.

How to use this executor?

Choose a runtime, open or create a file, write your code, and select Run. Output appears in the terminal, while signed-in users can keep files in a workspace, install libraries, and collaborate through shares.

Select a version and edit the current file or open another workspace file.
Run the code, provide input in the terminal when requested, and review saved run history.
Use libraries, workspace tools, sharing, and chat when your task needs more than one snippet.

Start coding

VersionSelect a runtime before running. Installed libraries belong to the selected language version.
Open files lists unopened text files. Select one to open it in an available editor tab.
Run executes the active file with its workspace available to the program.
Stop ends the current execution before its time limit.
Advanced Mode opens the IDE layout with a file explorer and movable, resizable terminal.

Workspace and libraries

New file creates a file in the selected parent directory and opens it in the editor.
New folder creates a directory, up to the supported nesting depth.
Upload files adds one or more safe files to a directory.
Download saves a file or a zipped directory to your device.
Copy duplicates a workspace file or directory.
Rename changes a file name and updates its open tab and history.
Move relocates an item; files and directories can also be dragged.
Delete removes an item after confirmation and closes its open editor tab.
Pre-installed Libs shows packages already included in this runtime.
Add Lib installs a private package for your account and selected runtime.

Terminal and assistance

Run history opens previous and next saved outputs.
Copy output copies the current terminal text.
Clear output removes all saved run outputs after confirmation.
Explain summarizes the active code.
Recents opens recent explanations and results.
Settings changes fonts, themes, and visible toolbar buttons.

Sharing and collaboration

Share publishes the current file, a directory, or the whole workspace as a snapshot or live workspace. You control link access and who may save changes.
Logs records saved collaborator actions in a shared workspace.
Share chat lets signed-in participants message, mention people, share code blocks, and see online or typing members.
Live collaboration shows collaborator cursors and saved file updates. View-only users can load the latest version or keep temporary edits.

Quotas and limits

Execution time is limited to 20 seconds per server run. Use Stop when you want to end it sooner.
Workspace storage allows 100 files, 100 directories, and 10 MiB total for each signed-in user workspace and language.
Files and folders are limited to 1 MiB per file and 5 nested directory levels. Files generated by your code follow the same quota.
Private libraries are limited to 10 installed packages for each account and runtime.
Share snapshots can contain up to 100 files, 1 MiB per file, and 2 MiB total.
Share invitations support up to 25 people per share, with access and saving permissions managed separately.
Share chat allows 2,000 characters per message and 10,000 messages per share.
Quota enforcement also applies while code is running. A write that exceeds file count, size, storage, or nesting limits is rejected safely.

Good to know Only run code and install packages you trust. Execution is isolated and limited, but your program can still modify files inside a writable workspace up to its enforced quota.

💡 Swift Basics Guide for Beginners

1. Declaring Variables and Constants

Use var for variables and let for constants in Swift.

var age = 30
let pi = 3.14159
var name = "Alice"
let isActive = true

// Constants cannot be reassigned
// pi = 3.14 // ❌ Error

2. Conditionals (if / switch)

Use if, else if, else for logic, and switch for multiple branches.

let number = 2
if number == 1 {
    print("One")
} else if number == 2 {
    print("Two")
} else {
    print("Other")
}

switch number {
case 1:
    print("One")
case 2:
    print("Two")
default:
    print("Other")
}

3. Loops

Swift supports for-in, while, and repeat-while loops.

for i in 0..<3 {
    print(i)
}

var n = 3
while n > 0 {
    print(n)
    n -= 1
}

4. Arrays

Arrays store ordered collections of values.

var numbers = [10, 20, 30]
print(numbers[1])

5. Array Manipulation

Use array methods like append and removeLast.

var nums = [1, 2, 3]
nums.append(4)
nums.removeLast()

for n in nums {
    print(n)
}

6. Console Input/Output

Use print() for output. Input requires additional work in Swift Playgrounds or Xcode.

let name = "Alice"
print("Hello, \(name)")

7. Functions

Define reusable functions with parameters and return values.

func add(a: Int, b: Int) -> Int {
    return a + b
}

print(add(a: 3, b: 4))

8. Dictionaries

Dictionaries store key-value pairs.

var ages = ["Alice": 30]
print(ages["Alice"] ?? 0)

9. Error Handling

Swift uses do-catch for error handling.

enum MyError: Error {
    case runtimeError(String)
}

do {
    throw MyError.runtimeError("Something went wrong")
} catch let error {
    print(error)
}

10. File I/O

Swift uses FileManager or String methods for file I/O.

let text = "Hello File"
try text.write(toFile: "file.txt", atomically: true, encoding: .utf8)
let content = try String(contentsOfFile: "file.txt")
print(content)

11. String Manipulation

Use methods like count, contains, uppercased(), etc.

let text = "Hello World"
print(text.count)
print(text.contains("World"))
print(text.uppercased())

12. Classes & Objects

Swift supports OOP using class and init.

class Person {
    var name: String
    init(name: String) {
        self.name = name
    }
    func greet() {
        print("Hi, I'm \(name)")
    }
}

let p = Person(name: "Alice")
p.greet()

13. Optionals

Swift uses ? and ! to handle optional values.

var name: String? = "Alice"
print(name ?? "Unknown")

if let unwrappedName = name {
    print(unwrappedName)
}