1. Feedback is gold

It's a catch-22. You need user feedback to build your MVP, but it's tough for someone to conceptualize how your non-existent app could help them, so you need to create an MVP to show them. User feedback is the best roadmap for your app, so it's best to seek it out early and often. This fact will drive many of the other decisions we make when developing an MVP. It's a cycle to clarify an idea from a hunch, to a list of pain points, to a list of requirements, mockups, and finally the MVP features in code. The more accurate of a starting point you have the better, so reach out to prospective users early.

2. Don't skip mockups

Before writing a line of code, you should have a clear visual representation of the app. The fidelity of the mockups isn't super important in my opinion. The real power is that the mockups are relatively quick to make and adjust and can be shared easily to facilitate communication between stakeholders. Everyone has assumptions, and it's so much easier to address conflicts early when you can easily drag boxes around.

Molding fuzzy ideas into boxes and buttons will bring up critical, unforeseen questions. It's worth spending time on mockups to discover these, rather than when you're deep in the code. These issues can be fixed by moving a box around or changing a layout in a drawing, rather than refactoring a large section of the app which wastes time and could lower quality since this critical item is being shoehorned in. Requirement changes get more expensive at every stage of development. The more that has been built, the more that may need to change.

If you're building MVPs often or need a high-fidelity mockup, I'd recommend learning Figma. Being able to import components from your chosen UI library and modify layouts and spacing easily is pretty handy. Draw.io is also a great tool if you want something simpler to drag and drop boxes around.

3. Choose a framework that lets you ship quickly

When you're building a first version, speed matters. Getting to a point where you can show off the app to stakeholders and receive feedback is the primary goal. You don't want to spend months building something and realize it doesn't meet users' needs. Better to find that out as soon as possible if that's the case.

AI-assisted development has become the norm, so pick a framework that AI models write good code with. At the time I was building Unison, React was clearly the strongest choice. I use both React and Svelte, and while I prefer Svelte, I noticed AI models occasionally producing outdated or made-up Svelte syntax. Since leveraging AI is mandatory for fast development speed, I went with React, specifically Next.js hosted on Vercel. Test how models perform with your chosen framework, since they're improving and changing rapidly.

4. Use a backend-as-a-service

Building and maintaining a custom backend from scratch adds a lot of complexity and risk to your MVP timeline. Backend-as-a-service (BAAS) platforms bundle hosting, database, APIs, auth, and other services in one place, letting you focus on the custom business logic that actually delivers value to users.

I used Supabase for Unison. It provides a hosted PostgreSQL database with a user-friendly JavaScript client, and CRUD APIs are included out of the box. That means no time spent developing, testing, and debugging common read/write operations and no server setup. And if the app takes off, the underlying infrastructure scales without a full rebuild.

5. Skip frontend unit tests — for now

This is a deliberate tradeoff, not an oversight. In an early-stage product, the frontend changes too frequently to justify the cost of maintaining tests. Wait until it stabilizes.

Bugs that frontend unit tests would catch, like a component not rendering or props handled incorrectly, are easy to spot quickly through manual testing. Your time is better spent shipping.

That said, I'd still recommend unit tests for backend functionality, since that's where your core logic lives. Our initial users are doing us a favor by testing an early-stage app, and we don't want to waste their time with bugs in the most important features. BAAS platforms like Supabase help here too. Their APIs are already tested, so you only need to cover your custom backend behavior, like Supabase Edge Functions.

6. Don't rush estimating

My toxic development trait is that often I hear a feature request and think it'll be straightforward, even if I haven't done it before. It's easy to get caught in the excitement of building something new and then forget that it needs to play nice with all of the other, slightly less shiny features. A lot of the pitfalls don't come from the feature itself, but how it fits into the existing system. Scoping a feature in a vacuum is a recipe for regret.

I recently added Google authentication to Unison and underestimated the complexity. Connecting to Google was simple, but refactoring the existing login and signup logic blew up the scope. The signup page became too cramped on mobile and needed to be split into two steps, which then introduced the problem of half-completed profiles. This spurred some discussion about the step order and overall messaging for the best UX.

Before giving an estimate, think through edge cases and which other parts of the app will need to change. And as always, add a buffer for unforeseen issues bound to arise.

7. Beware of scope creep

More features sound good on paper, but it comes at a cost. It obviously takes more work to build new features, and there is a higher risk if features are added to the scope mid-project. There may not be sufficient time to properly plan the work before the deadline, leading to lower quality work or pushing the deadline out.

Creating a plan that locks down the major requirements early, and treating new major features as a separate decision with its own cost and timeline is the best way to handle this. Not all feedback is scope creep. The goal during an MVP build is to receive feedback and iterate on the app. That feedback could come from the users, the business owner, or other stakeholders. This is what we want.

Feedback about the major features already in development can typically be included, but additional features outside of the plan should prompt a discussion about how to handle the expanded scope. Maybe you push out the timeline, add it to the next phase of the project, or swap out a lower priority feature. Scope creep didn't become an issue in my project, mainly because we had a plan in place. It gave us a framework to communicate the constraints of the project when new ideas started to enter the picture.

Conclusion

There are many ways to build a software MVP, and hopefully some of these ideas resonated. The main idea across all of them to get feedback as fast as possible. If you disagree with any of my tips, leave a comment and I've love to discuss it.

In this post, I'll share the steps I took to upgrade to Svelte 5 and what I learned along the way. Here is the official migration guide for all the details on the upgrade.

Steps

1. Create a New Git Branch

git switch -c svelte-5-migration

2. Use the Svelte 5 Migration Script

This script prompts you to choose which directories to migrate, automatically converts Svelte components in those directories to the Svelte 5 syntax, and updates the core dependencies in your package.json.

npx sv migrate svelte-5

At first, the script didn't execute correctly and warned me that I had invalid HTML in my project. Apparently, wrapping an <a> tag around a <tr> tag isn't a valid way to add a hyperlink to a table row. 🤔 I thought this was interesting as it always worked and I hadn't been warned before, but I went ahead, refactored, and committed the changes to clear my working directory. After that, the script ran successfully.

Note - You can also try migrating one component at a time in VS Code from the command palette Cmd+Shift+P with the command Migrate Component to Svelte 5 Syntax to get more control. I couldn't get this to work for me and went with the script, but it's worth looking into.

3. Update Dependencies

npm install

It's necessary to run npm install afterwards to update dependencies based on your new package.json to ensure everything still plays nice together. I got some errors saying I needed to update other packages so I did, but that didn't resolve the errors right away. I deleted my node_modules directory and my package-lock.json , then ran npm install again so they could be built from scratch, and that fixed my issues.

4. Review Updated Components

After my components had updated, VS Code still thought the new Svelte 5 syntax was invalid for some reason. I restarted VS Code and the errors went away, so you may need to do the same.

Here is part of my ResetPassword.svelte component script tag before the migration.

let email = ''
let showEmailValidation = false
let error = ''
let loading = false
let success = false
let buttonRef: HTMLButtonElement

$: disabled = loading || showEmailValidation
$: if (buttonRef && !showEmailValidation) {
    buttonRef.disabled = false
    buttonRef.focus()
}

And here it is after

let email = $state('')
let showEmailValidation = $state(false)
let error = $state('')
let loading = $state(false)
let success = $state(false)
let buttonRef: HTMLButtonElement = $state()

let disabled = $derived(loading || showEmailValidation)
run(() => {
    if (buttonRef && !showEmailValidation) {
        buttonRef.disabled = false
        buttonRef.focus()
    }
})

The new Svelte 5 rune syntax has been added so the code more explicitly shows our state with the $state() rune, derived state with $derived(), and effects with run. The $: syntax previously allowed for both derived state and side effects to run reactively when their dependencies changed, but if the migration script couldn't reliably migrate the statement to $derived(), it replaced it with run instead.

This is a stopgap solution that mimics the behavior of $: as it will run once on the server. run may look new, but it's technically deprecated as it is meant to be manually refactored during the migration. $: usages can usually replaced with one of the following

• $derived() - for defining a new reactive state from other state

• $effect() - for running side effects based on the effect's dependencies on the client only

• onMount() - for running code once when your component has been mounted on the client

Any state that depends on other state should use $derived so it can be rendered on the server. You should try to avoid using $effect() to update component state as it will look different on the client and server. Using effects to update state is a common pitfall in React as well, as it is often possible to define derived state instead. For code that just needs to run one time once it's on the client, prefer to use onMount over $effect().

Rich Harris, the creator of Svelte, even said the $effect() rune was named that way to discourage people from using it, as many developers are aware of the pains of useEffect from React (see this post). Effects in component frameworks are necessary but can be notoriously tricky to debug, so it's important not to overdo it.

5. Refactor run Statements

I started going through each file and refactoring the run statements. Thinking in terms of the new runes made it easier to reason about what should use $derived() and what I really need $effect() for. Here is an example.

I am loading whether a tournament’s leaderboard is toggled on from the server first, then I am transitioning control over to the client-side Svelte stores $isAuth and $isLeaderboard once the client is loaded.

// original
let combinedIsLeaderboard = serverIsLeaderboard
$: combinedIsLeaderboard = $isAuth && $isLeaderboard

After the migration script, this is a little more clear that we are instantiating a stateful value with server data and then updating it client-side.

// migrated
let combinedIsLeaderboard = $state(serverIsLeaderboard)
run(() => {
    combinedIsLeaderboard = $isAuth && $isLeaderboard
})

I could easily replace run with $effect() and call it a day, but this code starts to smell. After the migration, it's pretty clear that I'd be using $effect() to update component state, which I just mentioned is an anti-pattern. It's possible to derive the state directly from server data and client state, so $derived() is best choice.

We can use the Svelte browser variable to determine whether to use the client or server variables during the page load. It would also be possible to use onMount to pass control to the client, this is just my preference.

// refactored
let combinedIsLeaderboard = 
    $derived(browser ? $isAuth && $isLeaderboard : serverIsLeaderboard)

The result with $derived() is much more clear than the original code.

6. Resolve Miscellaneous Errors

This section is a bit of a tangent, but the Svelte VS Code extension and TypeScript seem to have gotten smarter with Svelte 5. Runes provide more clarity in the code, and they also provide more clarity in error messages.

$state()

The migration did a perfect job converting the original state assignments to the new $state() syntax. However, if I never initialized the state in the first place, it's now pretty obvious that those errors are my fault.

// original
let buttonRef: HTMLButtonElement

// migrated
let buttonRef: HTMLButtonElement = $state()

Now that we're required to use $state(), TypeScript read my initial state as undefined and complained since buttonRef is expected to be a HTMLButtonElement. This is a good thing because this issue existed before the migration, it just wasn't obvious to me (or TypeScript, for that matter).

// refactored
let buttonRef: HTMLButtonElement | null = $state(null)

Now TypeScript is happy, and we just need to explicitly check if the buttonRef exists later in the code.

bind

Here’s another hidden issue that already existed in my code. The bind keyword is used to sync data in both directions between a child component and state on a parent. VS Code now pointed out that I was incorrectly using it when passing data one way from parent to child, where it isn't necessary.

// incorrect
<FormError bind:error />     // shorthand for bind:error={error}

// correct
<FormError {error} />        // shorthand for error={error}

Be sure to check your console for warnings as well. I got a warning on one component that I was "Binding to a non-reactive property". Once again, it wasn't lying.

// incorrect
<input type="hidden" name="slotId" bind:value={slot.id} />

slot.id is a property on a prop being passed into my component, not a stateful value that needs to be bound to the child input's value. Just passing it one way to the input is sufficient.

// correct
<input type="hidden" name="slotId" value={slot.id} />

These new errors were confusing at first but helped me understand Svelte better. I digress.

7. Verify Migration Changes

Slots → Snippets

Slots have now been replaced with a new concept called snippets. I didn't need to make any manual changes for this update but it's worth checking out on your end.

Snippets are reusable pieces of code inside your component. They can be passed to other components as props. Any content inside the component tags that is not a snippet declaration implicitly becomes part of that component's children prop. If you're familiar with React, it's like when a React component written as the child of another component becomes part of the parent component's children prop.

Here's an example from the Svelte docs.

// App.svelte
<Button>click me</Button>

The string click me becomes a snippet that is passed to the Button component as the children prop. The new @render tag can then render snippets in the markup.

// Button.svelte
<script>
    let { children } = $props()
</script> 

<!-- result will be <button>click me</button> --> 
<button>{@render children()}</button>

So instead of content being passed into a slot, it's defined as a snippet, passed as the children prop, and rendered with @render.

Also note the new $props() rune for defining component props more explicitly. I never liked the old export syntax, because we're importing these props into the component right??

Anyways...

This is a simplified version of what happened in my root +layout.svelte component.

// original
<script>
    import type { RootLayoutData } from '$lib/types'
    export let data: RootLayoutData
// ...
</script>

<slot />
<footer />

// migrated
<script>
// ...
    import type { RootLayoutData } from '$lib/types'
    import type { Snippet } from 'svelte'

    interface Props {
        data: RootLayoutData
        children?: Snippet
    }

    let { data, children }: Props = $props()
// ...
</script>

{@render children?.()}
<footer />

More complicated? Maybe. But snippets seem pretty useful so I like them so far. Be sure to check your components for other changes made by the migration.

Unit Testing

Writing unit tests for this project was a slog since I neglected them at first, but this was a moment I was thankful to have them. After the migration refactored almost every component automatically, being able to run npm test and find issues quickly was very handy. My unit tests pointed me to the problematic components and gave me confidence that everything worked once all the tests passed.

I also ran the app locally to see it with my own eyes. Nothing changed, and it was beautiful.

8. Merge Branch Back Into Main

git switch main
git merge svelte-5-migration

...or create a pull request if that's what your team does. You're done!

Takeaways

I didn't know what to think about runes at first but now I'm a big fan. The $: was cool but maybe a little too cool. 😎 Replacing $: with $state(), $derived(), and $effect() makes it easier to use the right tool for the job. I would highly recommend upgrading when you have the chance.

Component frameworks are meant to hide complexity so developers can focus on the custom app logic. Determining how much to abstract away is both an art and a science that framework authors think about. More abstraction usually means less complexity for the developer but also less control.

Frameworks exist on a spectrum of abstraction and in my opinion Svelte 4 was pretty high on the list, at least compared to React or Angular. Features like $: and the export props syntax now feel too abstracted and "magical" as they're doing a lot under the hood.

Svelte 5 turns the abstraction dial down and adds more clarity to the syntax, which feels a lot more natural to me. This update is a big improvement on my favorite framework and I can't wait to dig into it more.

Back in 2016, I started my career as a software consultant for a healthcare tech company. I worked with customers to learn about their needs and then configured their business logic into our product. We used a browser-based interface to enter these settings, and it often involved a LOT of mundane, repetitive work. Think checking hundreds of checkboxes or filling out a form hundreds of times. My muscle memory got pretty good, but my poor fingers could only move so fast. It was a grind to get through it once I had planned out what needed to be done.

During our initial training, we were learning how to use the app and there was a page that required us to enter a default number into many fields. Our instructor told us to create a new browser shortcut and shared a long line of code that started with javascript: as the URL. Hmm 🤔, that doesn't look like a website... When I clicked the shortcut, it immediately filled out every field with a default value, and I was amazed. I was already excited to start my job, but now I was even more enticed by this sorcery I just witnessed.

I had taken a Matlab coding class in college, but dry engineering homework didn't really hit the same way as seeing manual work disappear in front of my eyes. I had to put my curiosity on the back burner for the first few months so I could get settled learning the product and the industry, but I knew I would be revisiting this magic string of text.

The company was growing rapidly, and lots of new work was coming in. It was an exciting place to be but also very busy. My teammates and I started working longer hours to stay afloat, and we needed a way to implement projects more quickly. There were a few other consultants who knew JavaScript and tinkered with creating new scripts, so I asked them to teach me.

I learned that they were making bookmarklets, or snippets of client-side JavaScript packaged as a browser bookmark. These bookmarklets had access to everything in the browser, and had the power to manipulate page elements however we wanted. My teammates showed me the basics, then on the weekends I studied Eloquent JavaScript with the goal of making my own.

All About Bookmarklets

Web apps store data in a database on a server, but they often need to store some data in the user's browser after the page is loaded so the page has quick access to it. In a modern framework, the client data can be tricky to access from outside the app, but this app was old enough that it used the browser's global scope to hold all the app's client-side variables. This meant all the client-side variables were available directly from the browser console and by extension, bookmarklets. Bingo.

Because bookmarklets only have access to client-side data, we used them to augment our own client-side human actions like filling out input fields, submitting forms, and navigating the app. We weren’t able to edit the backend, APIs, database, etc.

Our bookmarklets all had a similar style, though some were more complex than others. Some common themes were -

• We wrapped everything in an IIFE (Immediately Invoked Function Expression), allowing us to define and execute our logic as soon as the bookmarklet was clicked.

• We used prompt() to collect user input, such as the number to populate in multiple text fields.

• We implemented basic input validation was to ensure correct data entry, and used alert() to notify users of any errors.

• For more complex DOM element selection, we occasionally relied on JQuery.

Here is an example of a JavaScript snippet that prompts the user for a number, then fills that value into every number input field on the page. It also has some validation to ensure a number is entered.

(function() {
  const number = prompt('Enter a number to fill into all number fields:');
  if (number !== null) {
    if (isNaN(number)) {
      alert('Please enter a valid number.');
    } else {
      const inputs = document.querySelectorAll('input[type="number"]');
      inputs.forEach(function(input) {
        input.value = number;
      });
    }
  }
})();

During development we would copy and paste our code from a code editor into the browser console to test it out. Once the script was finished, we would minify it to reduce it's size for sharing. Then we’d add javascript: in front to tell the browser to execute the code rather than redirect to a website, and finally we’d create the browser bookmarklet with the minified script as the URL. The final code would look something like this.

javascript:!function(){
const e=prompt("Enter a number to fill into all number fields:");
null!==e&&(isNaN(e)?alert("Please enter a valid number.")
:document.querySelectorAll('input[type="number"]')
.forEach(function(l){l.value=e}))}();

I was hooked!

This was the perfect playground to learn how to code. The most important part was that I was naturally incentivized to build useful tools so that my teammates and I could finish our work faster and go home on time. We built scripts to -

• Fill in many input fields

• Check many boxes/radio buttons

• Submit forms

• Import/export data

• Filter items on the page

• Navigate to other pages

• and much more

Eventually we created custom keyboard shortcuts with a special Chrome extension to increase our efficiency even more. All of this made learning how to code absolutely addictive because I could quickly whip something together and save my team of many hours of menial labor.

I don't think I've ever felt more powerful than the moment my first script executed properly. Now I could make our product do (almost) whatever I needed it to do, without anyone else's approval. I've been chasing that dragon ever since. 🐉

Turmoil Brews

More and more consultants became proficient in JavaScript and created scripts for their own use cases. We started to share these with each other and sometimes jokingly bartered them on a "black market" in return for favors around the office like covering the support phone or the kitchen cleanup shift. The black market provided yet another source of motivation to continue developing new scripts.

The company was aware of a few approved scripts like the ones we received in our training, but they were not onboard with all of these new ones popping up every week. They were concerned for a few reasons -

• Having the ability to solve our own problems was great for the short term, but it concealed necessary product enhancements from our development team so they were not able to implement these features into the product.

• Critical project timelines were relying on hobbyist code.

• There was always a risk of a consultant sharing this code with a tech-savvy customer who could later demand support for these unsanctioned features. This would have been the cardinal sin, and never happened as far as I'm aware.

We knew their concerns, but the scripts made us so much more efficient and our project timelines were now relying on them. It would have been impossible to hit our deadlines without them.

Where Are They Now?

Eventually the company documented all of these scripts and prioritized building them into the product as first-class features. The app has since been fully rebuilt with modern tech and no longer puts everything into global variables in the browser, so the scripts aren't effective anymore. These scripts faded into the distance, relics of the past. Old-timers now talk of the good old days when you could imagine a feature, build it, and start using it that day.

I ended up with a decent understanding of JavaScript, but absolutely no other coding skills. As an activity during COVID, I tried freelancing with Google Apps Script, which is essentially just JavaScript with Google functionality added for managing G-Suite apps. It started off fine but quickly escalated in complexity, and I realized how in-over-my-head I was and had to quit. I had fallen off of the proverbial "Mount Stupid" of the Dunning-Kruger effect curve, headfirst into the "Valley of Despair".

It was a terrible feeling at first, and I felt disgusted when looking at code - like it was mocking me, and that I'd never be able to understand it. I took about a year off from coding before remembering how much I loved it in the first place, got started on a new project, and never stopped again.

My company required a Bachelors degree in computer science or equivalent experience to join their development team. Another degree would have taken me three years to complete while working, not to mention the cost. By this point I knew for sure I wanted to become a programmer and couldn't imagine waiting that long to make it a reality. At the time of writing this article, I’d still be working on that degree.

I tried learning on my own in the evenings but I wasn't sure where to focus. I needed outside help to cross the Valley of Despair, and found Carolina Code School, a full-time in-person bootcamp near me. After mulling it over for months, I took a leap of faith and quit my job to pursue a new career in software development. I completed the bootcamp and was fortunate to find a job before the post-COVID tech layoffs. This was the biggest risk I've taken in my life, but it felt like a bigger risk to do nothing.

As a budding developer building a portfolio of work, you won't have the luxury of having a product analyst to determine requirements or a sales team to sell your app. On top of learning new tech and learning how to code, right now you are also in charge of every other business role at your one-person software shop. The more familiar you are with each role, the more professional and competitive your portfolio will be.

In this article, I'm going to share some context about typical development-adjacent departments, and practical tips to make you stand out.

Departments

I realize it is incredibly arrogant to reduce all of the following fields to fit into the rest of this blog post, and that is what I'm about to attempt to do. Each one is deep enough to dedicate your entire career to, and I encourage you to explore each of them in more detail. Here are some starting points for you to incorporate them into your projects.

Product Management

The product management team is responsible for defining what features the product should have. I previously worked as a product analyst, so I have a bit more to say about this section than others. It is closely linked with development, and there is a lot to be gained by having an understanding of the product team. Day to day, it boils down to a few things.

Meeting with customers to determine pain points

Code is just a tool to solve problems, and the first step in the lifecycle of a software project is the problem itself. It can be difficult for a company to know exactly what the right problem to solve is, so it is common to have discussions with customers experiencing the problem regularly. Steve Jobs famously stated that you must start with the customer experience, and then work backwards.

Forming requirements for new features

Large features are organized into epics like "Create a user authentication workflow", which are comprised of smaller stories such as "Build a login page", "Build a registration page", and "Build a Forgot Password workflow". These stories are broken down into acceptance criteria like "Ensure a login with correct credentials is successful" and "Do not allow passwords of less than 10 characters during registration." Together, all of these stores and AC make up the requirements for the feature, which development will later use as a blueprint to build the product.

Prioritizing what features to build

...and maybe more importantly, what features to forget. Saying "No" is one of the most powerful tools in the product team's toolbox, because there are simply too many possible features to build with the the team's finite resources (people and time). Product teams use a backlog, which is just a list of all the stories and bugs they are responsible for, ordered by importance. They constantly reprioritize the backlog based on the current demand for each item to make sure the most important ones are being handled.

Tips

• When building a portfolio project, think about your hypothetical customer/user before you start writing any code. What is the main problem they would need to solve? What is the simplest way you can solve it?

• During your planning phase, write out some user stories for the most important features. They are from the perspective of a typical user and commonly take the form "As ___, I want to ___ so that I can ___." It's a good exercise to get you thinking about your project from the perspective of someone using it, who in real life could be someone who hires you!

• Create a backlog and any time you have an idea for a new feature or find a bug that needs to be fixed, add it to the list. If it is the most important, put it on top. If it isn't critical, put it lower. Determine what is absolutely necessary for your MVP (minimum viable product) and deprioritize the rest. Once your MVP is finished, feel free to polish it up with the remaining items from the top of your backlog.

User Experience (UX)

The UX team focuses on enhancing user satisfaction using the product. They design interfaces that focus on solving the users' problems in a way that would be most natural. They research market trends to find what would make their product more engaging. Usability, accessibility, visual design, and many other factors go into building high-fidelity mocks of user interfaces. They share these mocks with the product and development teams, then iterate upon them before development brings them to life.

Tips

• Make mocks of your screens before coding. They can be hand-drawn wireframes, basic shapes in a tool like draw.io, or something higher fidelity like Figma. It saves a lot of time in the long run to figure out the layout and flow of your app before starting to code.

• Make your app mobile responsive, and actually test it out on a real device. People scrolling on LinkedIn may pop it open on their phone, so it should look good! Most UI libraries provide tools to make responsiveness easier than raw media queries.

• Be intentional with the visual design. Spending a little time learning about color theory, typography, negative space, and other UI concepts can go a long way. Pick a color palette and a font, and find a UI library to handle basic components.

• Make the app as easy to use as possible. Reduce clicks, offer some functionality without a login if possible, and group related components near each other.

• Develop an eye for "jank", and remove it. If your app has components moving around that shouldn't, requiring more clicks than expected, or generally doing things that might raise an eyebrow, find out why and resolve it. If your app doesn't have any noticeable flaws, the important parts will shine.

Quality Assurance (QA)

The QA team is responsible for ensuring that the product meets the requirements set forth by the product team, and is free of bugs. They are the final human check before the product is released into the wild. QA conducts both manual and automated testing to simulate typical user behavior as well as more unlikely edge cases to give the company confidence in all code changes. In a larger codebase, a code change can affect something in a seemingly unrelated area, so the more automated testing you have, the more likely these sneaky bugs will be caught.

Unit tests are technically a developer's responsibility since they are part of the code, but I've included it in this section because it's another tool that professional teams use to test code. Unit tests allow you to test the behavior of a single "unit" of code against predefined test cases by simply running a test command.

Instead of preparing a fake scenario and manually calling an API endpoint to test something, you could write a unit test loaded with mock data, see its response to that data, and assert that its output equals the value you expect. If anything in the codebase changes that affects the API endpoint, the unit test will fail when you run it and you've now located a bug that you may not have known to check for. It's important to test the behavior of your code as a user might interact with it, rather than internal implementation details.

There is a lot of debate about how much unit test coverage is sufficient, as it may not be practical to write a unit test for every single thing. A common number I've seen to strive for is 80% coverage. Having a high coverage allows you to be more confident in your code changes and adapt to changing requirements. Test-driven development (TDD) takes this to the next level and recommends that you write unit tests before you write any code, watch them fail, and then write the code to make them pass. TDD is beyond the scope of this article, but it's definitely something to look into.

Tips

• Be your own QA team. After you add a new feature, check other related features to ensure they behave as expected. Confirm that they fulfill the requirements you created previously when you had your "Product" hat on.

• Think of edge cases and try to break your app through user interactions. What happens when you add letters to a field expecting a number? What happens if you click a button very quickly many times?

  • Prevent your users from getting into a bad situation. Use form field validation, API endpoint permission rules, disabling a button when its action is being awaited, etc.

• Write some unit tests. While they aren't quite as critical as in a professional app, having a few unit tests on API endpoints and key frontend behavior will give you confidence in your code and in your ability to speak about testing in interviews.

Sales and Marketing

Sales and marketing are typically two closely related departments. The sales team is responsible for selling the product to customers and generating new revenue for the business. The marketing team drives product awareness, interest, and brand reputation. Their goal is to get prospective customers thinking about the product and connecting them with the sales team.

As you are developing your portfolio apps, always think about who you are serving and how you would reach them. An excellent portfolio project would have real users you can market to and maybe turn into a business, but that is not necessary to demonstrate your skills. You will, however, need to market your projects online at least via LinkedIn and Github so that someone looking at your profile can see what you have to offer.

Tips

• HOST. HOST. HOST. If at all possible for your app, get it hosted somewhere online so that anyone who wants to check it out can. Nobody is going to clone your repo locally and run it on their machine.

  • For static sites and frontend repos, look at Netlify, Vercel, or GitHub Pages. For backend and full stack repos, check out a PaaS like Render or Railway. At the time of writing these have a free tier.

• Write good README's for your GitHub repos. You've done all the hard work, now let the world in on what you've been up to. Without a README.md file, your repo is just a folder with some code. In your README, add an overview of the project, tell people how to use it, list what technologies you used, and add any other relevant details.

• Post about your project on LinkedIn. You can post updates as you go, or just when it is completed and hosted.

  • Bonus points - make a video walkthrough of the project and share it. It's an easy way to demonstrate two of the most important things interviewers are looking for.

    1. You have the skills

    2. You can communicate with other humans

  • You can use Loom to record your screen and face easily, here is mine as an example.

Selling Yourself

No, I'm not talking about selling your soul to the devil or selling out to The Man. The bigger picture of marketing your portfolio projects is to make a strong showing for yourself on the job market. When you are job searching, you are selling your skills and experience to buyers - companies, startups, freelance clients, etc.

Tips

• Build a network so that people can find you. Connect with people on LinkedIn including other developers, people in related fields, or anyone who seems interesting. Go to in-person meetups to make local connections with other developers and techies in these adjacent fields.

• Post regularly on LinkedIn. It lets your audience know that you are on the market and that you're continuing to sharpen your skills. Share updates about what you're learning, or when your project is ready to check out.

  • Add posts you are proud of to the Featured section on your LinkedIn profile so that your network can easily find your best work.

• Before an interview, think about what salary you could realistically ask for based on your skills, the position, and the market. Also know what number you'd be happy with if they negotiate, and know what your minimum is.

Conclusion

There are several other departments at tech companies like Site Reliability Engineering (SRE), Data Engineering/Analysis, and Customer Success, but I find that the ones listed are the most applicable to your journey. I know this is a lot on top of your normal studies, but the job market is full of junior developers who want the same thing that you do. Leveraging these tips is something within your control to give you an edge. As a side note, many people assume that they must be a developer to get into tech, but you can target one of these other fields instead if they seem like a better fit. Good luck!

Web scraping is tough. From getting blocked by websites to parsing incomplete data, many roadblocks can slow you down, reroute you, or stop you entirely. ScraperAPI removes the headaches so you can focus on your project's big picture instead of the intricacies of web scraping.

What is ScraperAPI?

You guessed it - it's an API service that makes web scraping a breeze. Rather than requesting data from websites directly, you can route all of your requests through ScraperAPI's endpoints to take advantage of their rich feature set. It acts as a middleman to make web scraping requests for you so that you won't be blocked and then returns the correct data in various formats.

Here's a quick example using Python's requests package. Let's scrape a page I built with React. Traditional websites render HTML on the server, so requests to scrape them will return the full HTML you're looking for. Many modern websites use popular frameworks like React to render it in the browser after your request is made, meaning the HTML from the server will essentially be empty at first. ScraperAPI has a feature to render the HTML for you before returning it, making it super simple to access the data you need.

Without ScraperAPI -

import requests

url = 'https://state-management.willbraun.dev'

r = requests.get(url)
print(r.text)

Response

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8" />
        <link rel="icon" type="image/svg+xml" href="/vite.svg" />
        <meta name="viewport" content="width=device-width, initial-scale=1.0" />
        <title>State Management Patterns</title>
        <script type="module" crossorigin src="/assets/index-8d0aa6b9.js"></script>
        <link rel="stylesheet" href="/assets/index-cda38e90.css">
    </head>
    <body>
        <div id="root"></div>
        // NOTE - No content is loaded yet!
    </body>
</html>

Notice the body of the response has one empty element - not much to work with! Let's try using ScraperAPI, with JavaScript rendering enabled. I'll be using the payload parameter of Python requests to build the full URL to send to ScraperAPI, and I've stored my API key in an environment variable.

import requests
from decouple import config

url = 'https://state-management.willbraun.dev'
payload = {
  'api_key': config('API_KEY'), 
  'url': url, 
  'render': 'true', 
}

r = requests.get('https://api.scraperapi.com', params=payload)
print(r.text)

Full URL

https://api.scraperapi.com/?api_key=<your_api_key>&url=https://state-management.willbraun.dev&render=true

Response

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8">
        <link rel="icon" type="image/svg+xml" href="/vite.svg">
        <meta name="viewport" content="width=device-width, initial-scale=1.0">
        <title>State Management Patterns</title>
        <script type="module" crossorigin="" src="/assets/index-8d0aa6b9.js"></script>
        <link rel="stylesheet" href="/assets/index-cda38e90.css">
    </head>
    <body>
        <div id="root">
            <header>
                <nav>
                    <a href="https://blog.willbraun.dev/demystifying-state-management" target="_blank" rel="noopener noreferrer">Blog Post</a>
                    <a href="https://github.com/willbraun/state-mgmt-patterns" target="_blank" rel="noopener noreferrer">GitHub</a>
                </nav>
            </header>
            <p>OFF</p>
            <button>Toggle with Prop Drilling</button>
            <p>OFF</p>
            <button>Toggle with Context</button>
            <p>OFF</p>
            <button>Toggle with Zustand</button>
            <p>OFF</p>
            <button>Toggle with Redux</button>
        </div>
    </body>
</html>

Look at all that HTML! This problem is a thing of the past, and ScraperAPI's other features are just as easy to set up.

Features

Rotating IP Addresses

ScraperAPI automatically rotates through millions of IP addresses as it scrapes data, which provides you with a variety of benefits.

Many websites have measures in place to prevent scraping. ScraperAPI distributes scraping requests across many IPs and can automatically retry on failed requests, greatly reducing the likelihood of triggering IP blocks.

Some websites may impose rate limits on the number of requests that can be made from a single IP address within a specific time frame. Rotating IP addresses allows you to bypass these limits and scrape more data without being throttled.

JavaScript Rendering

This is the feature from our example. JavaScript running in the browser may be required to render the part of the HTML. It is more complicated to scrape as the initial response from the server is incomplete. You have to wait for the JavaScript to run after you send the request to the server.

By adding &render=true to your API call, ScraperAPI will handle this step for you. It will render all of the JavaScript on your desired page, and return the full HTML to you.

Auto Parse to JSON

HTML is typically returned from a scrape request as a long string of text. This may be fine for your use case since it is standard, but the next step is usually to transform the response into a more workable format so that you can find what you need.

By adding &autoparse=true to your API call, ScraperAPI will format the response as JSON if possible. One less step for you to worry about!

Geolocation

Sometimes websites show different data depending on what part of the world you are requesting data from. To show results for a particular location, you can specify the geolocation of the IP addresses used by adding the &country_code parameter to your API call. The available countries are listed in the ScraperAPI documentation.

Structured Data Endpoints

Trying to scrape Amazon or Google Search? ScraperAPI has you covered with prebuilt endpoints that return relevant data as JSON. This is a powerful feature that can really accelerate you towards your project goals. Here are the services currently available.

• Amazon Search

• Amazon Product Page

• Google Search Engine Result Page (SERP)

Let's check out the Amazon Search structured data endpoint, which returns Amazon search results from any search query. I was recently researching computer monitors, so let's see what we find with this method.

import requests
from decouple import config
import json

payload = {
  'api_key': config('API_KEY'), 
  'query': 'computer monitors',  
}

r = requests.get(
    'https://api.scraperapi.com/structured/amazon/search', 
    params=payload
)
parsed = json.loads(r.text)
print(json.dumps(parsed, indent=2))

Full URL

https://api.scraperapi.com/structured/amazon/search?api_key=<your_api_key>&query=computer+monitors

Response (shortened, it's a lot of data)

{
  "ads": [
    {
      "name": "SAMSUNG 32\" Odyssey G55A QHD 165Hz 1ms FreeSync Curved Gaming Monitor with HDR 10, Futuristic Design for Any Desktop (LS32AG550ENXZA)",
      "asin": "B09TMJ9LGR",
      "brand": "Samsung Gaming Monitors",
      "image": "https://m.media-amazon.com/images/I/81a+yL6ii9L.jpg",
      "has_prime": false,
      "is_best_seller": false,
      "is_amazon_choice": false,
      "is_limited_deal": false,
      "stars": 4.4,
      "total_reviews": 0,
      "url": "https://aax-us-iad.amazon.com/x/c/RPqPaOmO9TkKF7V6K9QDMy0AAAGLCkRA-gEAAAH2AQBvbm9fdHhuX2JpZDMgICBvbm9fdHhuX2ltcDEgICA-fF7_/https://www.amazon.com/gp/aw/d/B09TMJ9LGR/?_encoding=UTF8&pd_rd_plhdr=t&aaxitk=e43e49e9df54e5c84f02ddf50d96ae4a&hsa_cr_id=0&qid=1696684327&sr=1-1-9e67e56a-6f64-441f-a281-df67fc737124&ref_=sbx_be_s_sparkle_mcd_asin_0_bkgd&pd_rd_w=9Nsbn&content-id=amzn1.sym.cd95889f-432f-43a7-8ec8-833616493f4a%3Aamzn1.sym.cd95889f-432f-43a7-8ec8-833616493f4a&pf_rd_p=cd95889f-432f-43a7-8ec8-833616493f4a&pf_rd_r=9X10K8EY448WDY60KJJV&pd_rd_wg=lpoc4&pd_rd_r=93c1edaf-f358-4104-bbb9-6e608dc2b024",
      "type": "top_stripe_ads"
    },
    <more ads>
  ],
  "results": [
    {
      "type": "search_product",
      "position": 1,
      "asin": "B0773ZY26F",
      "name": "Sceptre 24-inch Professional Thin 1080p LED Monitor 99% sRGB 2x HDMI VGA Build-in Speakers, Machine Black (E248W-19203R Series)",
      "image": "https://m.media-amazon.com/images/I/81zM2vVM+wL.jpg",
      "has_prime": false,
      "is_best_seller": false,
      "is_amazon_choice": false,
      "is_limited_deal": false,
      "stars": 4.6,
      "total_reviews": 30226,
      "url": "https://www.amazon.com/Sceptre-E248W-19203R-Monitor-Speakers-Metallic/dp/B0773ZY26F/ref=sr_1_1?keywords=computer+monitors&qid=1696684327&sr=8-1",
      "availability_quantity": null,
      "spec": {},
      "price_string": "$89.56",
      "price_symbol": "$",
      "price": 89.56
    },
    {
      "type": "search_product",
      "position": 2,
      "asin": "B0148NNKTC",
      "name": "Acer 23.8\u201d Full HD 1920 x 1080 IPS Zero Frame Home Office Computer Monitor - 178\u00b0 Wide View Angle - 16.7M - NTSC 72% Color Gamut - Low Blue Light - Tilt Compatible - VGA HDMI DVI R240HY bidx",
      "image": "https://m.media-amazon.com/images/I/91K9SyGiyzL.jpg",
      "has_prime": true,
      "is_best_seller": false,
      "is_amazon_choice": false,
      "is_limited_deal": false,
      "stars": 4.7,
      "total_reviews": 15156,
      "url": "https://www.amazon.com/Acer-Frame-Office-Computer-Monitor/dp/B0148NNKTC/ref=sr_1_2?keywords=computer+monitors&qid=1696684327&sr=8-2",
      "availability_quantity": null,
      "spec": {},
      "price_string": "$99.99",
      "price_symbol": "$",
      "price": 99.99
    },
    {
      "type": "search_product",
      "position": 3,
      "asin": "B0BS9TDY31",
      "name": "Acer KB272 Hbi 27\" Full HD (1920 x 1080) Zero-Frame Gaming Office Monitor | AMD FreeSync Technology | 100Hz | 1ms (VRB) | Low Blue Light | Tilt | HDMI & VGA Ports,Black",
      "image": "https://m.media-amazon.com/images/I/81FTa3aSdnL.jpg",
      "has_prime": true,
      "is_best_seller": false,
      "is_amazon_choice": false,
      "is_limited_deal": false,
      "stars": 4.6,
      "total_reviews": 2158,
      "url": "https://www.amazon.com/KB272-Hbi-Zero-Frame-FreeSync-Technology/dp/B0BS9TDY31/ref=sr_1_3?keywords=computer+monitors&qid=1696684327&sr=8-3",
      "availability_quantity": null,
      "spec": {},
      "price_string": "$149.99",
      "price_symbol": "$",
      "price": 149.99
    },
    <many more search results>
  ],
  "explore_more_items": [],
  "pagination": [
    "https://www.amazon.com/s?k=computer+monitors&page=2&qid=1696684327&ref=sr_pg_2",
   <more page links>
  ]
}

More

See all features, including the following, in the full documentation here.

• Concurrent threads

• Automated retries

• Async requests

• Proxy ports

• SDKs

Let's get scraping!

There's no substitute for having the right tool for the job. Scraping without assistance can cost you valuable time searching for a usable site, wading through errors, and formatting data. ScraperAPI is a dead-simple way to level up your scraping game and put those challenges behind you.

ScraperAPI has a generous free tier that offers 5,000 API credits per month and paid plans that scale to your needs. They also offer professional support to assist you with your setup. The sign-up process is painless, and free plans do not require a credit card. I was able to start scraping with it in less than 5 minutes.

If only I had known about this when I built a scraping tool to simulate bets on tennis matches (link here). Since I was scraping websites directly, I was getting blocked and dealing with incomplete data. It would have been a much faster and smoother experience with ScraperAPI.

Originally, I wanted to find the absolute best solution for any project so that I could just learn that and continue building, but like many decisions in software development, the answer is - "It depends". In this post, I'll share the overarching patterns I've noticed along with technologies that implement them. Examples are shown for React, but there are similar solutions available for other frameworks.

I've created a functional demo for the examples shown in the following sections where you can follow along if you like. Here is the demo and Github.

What is State Management?

Many front-end frameworks use declaratively defined components that change over time. Components have an internal state that dictates their behavior at any given moment. A component's state does not live in a vacuum - it nearly always needs to communicate with other components' state data. Changing one component's state often requires updating many others in a domino effect, so managing and updating state is critical to controlling your app's behavior. As more components are added, it becomes more important to have a good state management solution in place.

Prop Drilling

As I mentioned, components have an internal state that can be controlled using built-in features. In React, a stateful value can be created inside a component using the useState hook. This value can be used to control the behavior of the component, such as conditionally rendering different outputs based on the state. That's great, but we usually need to share the state with other components so they can update in unison based on input from the user or database. Components can share state with child components (components rendered in their output) via the props object. The children can now use that value to control their own behavior and optionally pass it down to their children also.

This is a simple pattern that can work for small projects, but the main issue with this approach is prop drilling, which is not unique to React. If you have a complex app with many layers of components in the tree, and a low-level component needs the state of a high-level component, you will have to drill down from the top level and pass the state as a prop to every intermediate component along the way, even if those components will not use the value themselves.

This is an example to show the state in the Parent component being used by a Grandchild component. We have a simple toggle button that is controlling the state in the parent. Yes, this specific example could be made in a single component, but let's imagine this is a small part of a larger state where there are many settings controlled by many nested components.

// PropDrilling.tsx
import { useState } from 'react'

interface Props {
  isToggled: boolean
  handleClick: () => void
}

export const Parent: React.FC = () => {
  const [isToggled, setIsToggled] = useState(false)

  const handleClick = () => {
    setIsToggled(!isToggled)
  }

  return <Child isToggled={isToggled} handleClick={handleClick} />
}

export const Child: React.FC<Props> = ({ isToggled, handleClick }) => {
  return <Grandchild isToggled={isToggled} handleClick={handleClick} />
}

export const Grandchild: React.FC<Props> = ({ isToggled, handleClick }) => {
  return (
    <>
      <p>{isToggled ? 'ON' : 'OFF'}</p>
      <button onClick={handleClick}>Toggle with Prop Drilling</button>
    </>
  )
}

In order for Grandchild to have access to update the state, we pass isToggled and handleClick as props from Parent to Child, and again from Child to Grandchild. The props object argument in Child and Grandchild is destructured in this example to simplify access to the props themselves.

Complexity varies slightly depending on if you are using a framework with one-way data binding (React) or one with two-way data binding (an option in Svelte). Data flows one way in React - from parent to child, whereas in a two-way data binding framework, updates to the parent or child will update the other. With one-way binding, you also need to pass a function to update the parent state down to the child as a prop, as shown in the example. There are pros and cons to each method of data binding, but that is a separate discussion. The important part is that sharing data around the tree easily is commonly needed.

Each child that accepts props is logically linked to its parent, so prop drilling creates a chain of dependencies that quickly becomes tedious to maintain. The rest of the patterns show different ways of avoiding this problem.

Context API

In programming, the term "context" refers to the specific environment or set of conditions that code is run in. React has adopted this term for its Context API, which allows you to create a state context for your component tree. A context can be created at any point in the tree, then any child components that are wrapped inside the context provider will have access to the context's state.

Context is usually created at the top level so that the entire tree can have access to it. This bypasses the need for prop drilling between every intermediate component. The state still lives in the tree, it is just easier to access from the components.

Here is our previous example, rewritten using the Context API.

// Context.tsx
import { createContext, useState, useContext } from 'react'

interface ToggleContextProps {
  isToggled: boolean
  handleClick: () => void
}

const ToggleContext = createContext<ToggleContextProps>({
  isToggled: false,
  handleClick: () => ({}),
})

export const Parent: React.FC = () => {
  const [isToggled, setIsToggled] = useState(false)

  const handleClick = () => {
    setIsToggled(!isToggled)
  }

  return (
    <ToggleContext.Provider value={{ isToggled, handleClick }}>
      <Child />
    </ToggleContext.Provider>
  )
}

export const Child: React.FC = () => {
  return <Grandchild />
}

export const Grandchild: React.FC = () => {
  const { isToggled, handleClick } = useContext(ToggleContext)

  return (
    <>
      <p>{isToggled ? 'ON' : 'OFF'}</p>
      <button onClick={handleClick}>Toggle with Context</button>
    </>
  )
}

The toggle state and click function are still stored in the state at the Parent level, but they did not have to be passed down to Child or Grandchild as props. Grandchild was able to access them with the useContext hook.

Common use cases for this include global settings like color theme and current user account permissions. Context is accessible in React without any additional libraries, and it can be combined with the useReducer hook to add even more control to state updates.

More information on the Context API is shown here.

External Store

Another approach to sharing state without prop drilling is to separate the state from the tree entirely. A store is a home for your application's state outside of the component tree. Many libraries host state outside of the component tree and provide various ways of interacting with it.

Global variables are typically considered a bad practice in programming because they create dependencies between components using them, so what makes external stores different? Rather than merely creating a global variable that can be updated in any capacity from anywhere in the code, these libraries add guardrails so that state interactions are well-defined and testable. In Zustand and other libraries, you can create a store and define functions for updating it in a specific manner. There are still logical dependencies that would occur in any global variable scenario, but many find that the simplicity of this solution when storing complex data like your application state makes it worth it compared to the alternatives.

Here is our toggle, now written using Zustand.

// Zustand.tsx
import React from 'react'
import { create } from 'zustand'

interface ToggleState {
  isToggled: boolean
  handleClick: () => void
}

const useToggleStore = create<ToggleState>(set => ({
  isToggled: false,
  handleClick: () => set(state => ({ isToggled: !state.isToggled })),
}))

export const Parent: React.FC = () => {
  return <Child />
}

export const Child: React.FC = () => {
  return <Grandchild />
}

export const Grandchild: React.FC = () => {
  const { isToggled, handleClick } = useToggleStore()

  return (
    <>
      <p>{isToggled ? 'ON' : 'OFF'}</p>
      <button onClick={handleClick}>Toggle with Zustand</button>
    </>
  )
}

Notice how the state now lives in the store and not in the Parent component at all. The handleClick function uses Zustand's set function, which accepts the current state of the store and returns the updated portion of the state. Zustand merges the updated portion with the overall store, and the Grandchild component subscribes to the latest version of the state. See more information on this here.

Many state management libraries with an external store are inspired by our next pattern but with an emphasis on simplicity.

Flux Model

No state management article would be complete without the Flux model, which is almost synonymous with the Redux library for React. The Flux model also includes an external store to hold state outside of the component tree, but it uses a more structured, one-way data flow to manage state predictably. It consists of four key parts.

1. Actions - The different events or user interactions that can occur in an application.

2. Dispatcher - The central hub that receives actions and applies (dispatches) them to the appropriate store.

3. Stores - A store holds the application state and defines logic for handling actions and updating the state. There can be multiple stores, and they listen for actions from the Dispatcher and update their state accordingly.

   • Reducers are functions that accept the current state and incoming action and return the updated state. Reducers aren't explicitly defined in the Flux model, but the concept is still important here.

4. Views - This represents the user interface, specifically the components in the framework. They subscribe to changes in the store so the current state is always shown. Views are responsible for re-rendering content on the page when the state in a store is updated.

Here is our example using Redux, with the officially recommended Redux-Toolkit library as well. Redux-Toolkit is an abstraction over Redux with its own functions to simplify the configuration.

With Redux-Toolkit, a store is broken up into slices to divide the state into simpler pieces. First, we create a slice, set an initial state value, then add our actions and reducers. We have an action named toggle which corresponds to a reducer function that flips state.isToggled when the action is dispatched. Typically, the slice name would not have the word "slice" in it, but I wanted to make that more clear when it is used later. Reducer functions are automatically created by the createSlice function, and it is common to export those as the default export.

// toggleSlice.ts
import { createSlice } from '@reduxjs/toolkit'

export interface ToggleState {
  isToggled: boolean
}

const initialState: ToggleState = {
  isToggled: false,
}

const toggleSlice = createSlice({
  name: 'toggleSlice',
  initialState,
  reducers: {
    toggle: state => {
      state.isToggled = !state.isToggled
    },
  },
})

export const { toggle } = toggleSlice.actions
export default toggleSlice.reducer

Next, we configure the store itself. The store has a root reducer that aggregates the reducers from each slice.

// store.ts
import { configureStore } from '@reduxjs/toolkit'
import toggleReducer from './toggleSlice'

export type RootState = ReturnType<typeof store.getState>

const store = configureStore({
  reducer: {
    toggleSlice: toggleReducer,
  },
})

export default store

Finally, we have the components or views in the Flux model. The Redux Provider allows the store to be accessed by all nested components. Components can use the useSelector hook to specify what state to pull from the store. This subscribes components to the store so that as actions are dispatched and the state is updated, components will update automatically as well. We also import our toggle action from toggleSlice.ts so that we can dispatch it from our component.

// Redux.tsx
import React from 'react'
import { useSelector, useDispatch, Provider } from 'react-redux'
import store, { RootState } from './store'
import { toggle } from './toggleSlice'

export const Parent: React.FC = () => {
  return (
    <Provider store={store}>
      <Child />
    </Provider>
  )
}

export const Child: React.FC = () => {
  return <Grandchild />
}

export const Grandchild: React.FC = () => {
  const isToggled = useSelector((state: RootState) => state.toggleSlice.isToggled)
  const dispatch = useDispatch()

  const handleClick = () => {
    dispatch(toggle())
  }

  return (
    <>
      <p>{isToggled ? 'ON' : 'OFF'}</p>
      <button onClick={handleClick}>Toggle with Redux</button>
    </>
  )
}

export default Parent

The Flux model is a powerful way to manage state, and Redux is the choice for many enterprise React applications. Redux is overkill for our example, but it shows how much control you have at each step of the process. Also, Redux has a ton of additional functionality that is beyond the scope of this article. The robust structure does add complexity, so it's worth doing some research to determine if it is right for your project.

Conclusion

If you are searching for "what state management solution is the best", then you're going to get many different answers. It is common to use a combination of built-in component state where it makes sense and one of the other patterns I mentioned for more complex state. There are also technologies not mentioned that combine elements of these patterns in various ways. The best choice for you comes down to your application's business requirements and the other technologies in your stack.

Did I miss any patterns? Leave a comment and let me know!

I've been a tennis player and fan of the pro tour my whole life. Since sports betting in the US was legalized, I've seen some friends of mine make ridiculous bets on tennis matches. Even with my knowledge of the game and pro players, I can't stomach betting on matches as it seems too unpredictable. What if there was some way I could predict who was going to win matches well enough to win money in the long run?

Sure, this isn't a new idea. People have built elaborate sports betting models to predict outcomes well enough to make them rich, but it's difficult and risky. Casino oddsmakers have machine learning algorithms and an ocean of data at their disposal when setting betting lines, making it tough for amateur gamblers to win money over time. I'm always up for a challenge, so I formulated a plan to make smarter bets on tennis matches and dreamed of making easy money.

In this post, I'll share my plan and how it evolved, what technologies worked for me, and what I learned about them. I'll be discussing Python, Beautiful Soup, APIs, PostgreSQL, SQL, PLpgSQL, and launchd. This is not intended to be a step-by-step tutorial, but rather a way to show my decision-making through the entire stack.

Here is the GitHub Repository for this project if you'd like to follow along. Relevant code samples are included in this post.

My Master Plan

Step 1: Make a good bet

My core idea was simple, but why start complicated? To decide who to bet on for a given match, I needed two pieces of data:

1. The betting odds for an upcoming match

2. A prediction of who will win

American Betting Odds

American betting odds are based on a bet of $100, though you can bet any amount. Each player has odds associated with them to show how much you can win with your bet. There are odds for the money line, point spread, and many other bets. We are using the money line, which is only based on the win-loss outcome of the match.

• The favorite's odds are usually negative. Odds of -170 mean if you bet $170, you make $100 if that player wins (plus your original bet of $170).

• The underdog's odds are usually positive. Odds of 200 mean that if you bet $100, you make $200 if that player wins (plus your original bet of $100).

• Of course, if your player loses, you lose your bet!

Heavily lopsided matchups may have odds of -900/+700, while more even matchups may have odds of -150/+110. Casinos set these odds in such a way that they will probably make money no matter what, given a large enough number of people placing bets.

This information is already out there on the internet, I would just need to figure out the best way to bring it into my program to do the calculations.

Predicting The Winner

This part was a mystery for a while. At first, it seemed like I would need to build a separate program involving AI to determine this, which would have been a very deep rabbit hole. I stumbled upon a website called Ultimate Tennis Stats, which has a treasure trove of pro tennis data and built-in analytics.

They have a feature that allows you to set up a hypothetical tennis match between any two players, and it will output a probability of who will win as a percentage. For example, as of the time of writing, it predicts that Novak Djokovic has a 74.2% chance of winning against Rafael Nadal. Novak just won the Australian Open and Rafa has had some injuries lately, so this seems reasonable. This number is just what I needed to continue!

Credit to the Ultimate Tennis Stats team for building an amazing tool.

Putting It All Together

Remember, I'm not just betting on who the prediction says will win the match. I want to make bets that will win money over time factoring in the odds for both players. To do this, I imagined that the players would play 1000 matches, with the match results being split based on the predicted win probability. Then I'd calculate my payout if I bet $1 on player 1 for all matches vs player 2 for all matches. Whichever bet had the higher payout would be my choice. If both bets lose money, I would skip betting on this match entirely.

Here is a table to show the calculation. The light green fields are the inputs for each bet, and the player with the higher positive value in row 7 is my choice.

Step 2: Simulate bets

Before wagering a single cent, I wanted to ensure that this method would actually make money. I needed to simulate bets on real tennis matches as a risk-free proof of concept for my idea, rather than bet real money and find out the hard way.

In addition to the odds and win prediction before the match, I would also need to pull in the results of the match afterward so I could simulate a bet result in terms of dollars. If I bet $1 per match, a win on a player whose odds are +320 would earn me $3.20, while a loss would lose my $1 bet. As more matches are played, I would track my simulated earnings over time and store the results in a database for analysis.

Step 3: Automate bets with real money

If my hypothesis in step 2 is correct and the simulation shows a positive return, the next step would be to automate bets with real money, and get rich!

Time for Action

Designing the Simulation

Now that I had a plan for individual bets, I needed to build out my program to simulate them over multiple matches. The structure of the program would be simple - a single script file that would gather the data from the web, crunch the numbers, and output the results to a database. There would be no need for a graphical user interface. For these reasons, Python was an obvious choice of language to use. It is a multi-purpose, open-source scripting language with many packages that can be used to extend its functionality. Not to mention, the syntax is easy to read and it's a joy to work with.

There are two main objectives for the script

1. Create bet decisions for upcoming matches

2. Determine the simulated payout for completed matches

In my script, the updates come first as it made more sense to me to update completed matches before predicting future matches, but these sections could be run in either order.

Gathering the Inputs

I needed to reach out to the web to pull the odds and a win prediction into my program. This is a classic use case for an API (Application Programming Interface), a typical way for applications to communicate over the internet. However, many customer-facing casino odds APIs aren't free, and I still wasn't sure if this project would make any money. I also needed a way to get the win probability from Ultimate Tennis Stats.

This information is available publicly on the web, and after some research, I found that web scraping was what I needed. It's a programmatic way to pull in data from a human-readable web page. Web pages are made up of HTML, CSS, and JavaScript, where HTML represents the raw content of the page. Its main purpose is to provide a user with information after it is styled by CSS and made functional by JavaScript, but it can also be read by search engines for SEO.

Web scraping in itself is completely legal as long as the data is publicly available. However, there are some ethical issues you should consider to make sure your program isn't breaking other laws regarding personal or copyrighted data. For example, I wouldn't recommend scraping personal information to sell without permission. Check out this article for more information on ethical web scraping.

To recap, the three pieces of data that I needed for each match were

1. Betting odds for the match

2. A prediction of who will win

3. The results of the match

I had chosen my website for the prediction, so I went ahead to see if I could scrape it into my program. The most popular Python package for web scraping is called Beautiful Soup, so I installed it into my project directory. The magic is its ability to turn a long string of HTML as text into a BeautifulSoup object, which can be easily parsed with its built-in functions to extract the desired data.

Here is the web page with the probability I was trying to scrape.

This is a hypothetical match between the 2 top-ranked male players at the time of writing, but this could show any two players by manipulating the player IDs in the URL. I only need player 1's win probability as the other is dependent on it. Let's try to scrape this information with a demo program, demo.py, first without Beautiful Soup to get the raw HTML as text.

import requests

session = requests.Session()

p1_id = 4920 # Novak Djokovic
p2_id = 52602 # Carlos Alcaraz
url = f'https://www.ultimatetennisstatistics.com/headToHead?tab=hypotheticalMatchup&playerId1={p1_id}&playerId2={p2_id}'

response = session.get(url, headers={'User-Agent': 'Mozilla/5.0'})
print(response.text)

Here, we are using the built-in Python requests package to make a call to the URL of the web page. The session allows us to make the request and store additional parameters like the request header. Some websites block web scraping by checking the user agent, which has a default value similar to python-requests/2.25.0 depending on your version of Python. We need to change the User-Agent header to simulate a request from a browser so we are less likely to be blocked. I'm using Mozilla/5.0 but there are others that will work.

The IDs are Ultimate Tennis Stats' internal IDs for each player. I pulled them out of the URL as those will be determined dynamically later on depending on the upcoming matches. We can run this program by opening the terminal, cd-ing into our project directory, and running python3 demo.py. We get a long string of HTML as text, just as we were hoping. Great!

Now let's try parsing it with Beautiful Soup to find the probability. I used Beautiful Soup's find function to navigate the object and find where the text "Win Probability" is on the page, as I know the value isn't far away.

import requests
from bs4 import BeautifulSoup

session = requests.Session()

p1_id = 4920 # Novak Djokovic
p2_id = 52602 # Carlos Alcaraz
url = f'https://www.ultimatetennisstatistics.com/headToHead?tab=hypotheticalMatchup&playerId1={p1_id}&playerId2={p2_id}'

response = session.get(url, headers={'User-Agent': 'Mozilla/5.0'})
doc = BeautifulSoup(response.text, 'html.parser')
win_prob_row = doc.find(string='Win Probability')
print(win_prob_row)

But when I run it, I get a surprise 😱

None!? I can see the element on the page, so what's going on? After searching the HTML provided by that URL, the text "Win Probability" was, in fact, not there at all. So how was it showing up on the page?

The issue is that not all of the data is loaded immediately. The rest is being loaded from separate API calls after the initial page request. I found that it is common for web scrapers to request data from these API endpoints directly, rather than requesting just from the page URL. The response can come back as HTML, JSON, XML, or other formats.

You may be wondering - didn't I say earlier that using an API wasn't an option? I was referring to customer-facing APIs, where access is officially supported for building applications. The APIs that web pages use internally are not supported for external use (like web scraping) and can change frequently. Web scraping programs often require maintenance because the data can change without warning.

By checking the Chrome dev tools Network tab, we can find the API call the page makes to get the probability.

Bingo! The probability is in our sights now. The response format is HTML, so we can continue to use Beautiful Soup to parse it. Let's replace the URL currently in our code with the URL for the API (full URL shown in the Network > Headers tab).

import requests
from bs4 import BeautifulSoup

session = requests.Session()

p1_id = 4920 # Novak Djokovic
p2_id = 52602 # Carlos Alcaraz
url = f'https://www.ultimatetennisstatistics.com/h2hHypotheticalMatchup?playerId1={p1_id}&playerId2={p2_id}'

response = session.get(url, headers={'User-Agent': 'Mozilla/5.0'})
doc = BeautifulSoup(response.text, 'html.parser')
win_prob_row = doc.find(string='Win Probability')
print(win_prob_row)

Now we have the correct BeautifulSoup object, so we just need to do a little more manipulation to get the probability.

import requests
from bs4 import BeautifulSoup

session = requests.Session()

p1_id = 4920 # Novak Djokovic
p2_id = 52602 # Carlos Alcaraz
url = f'https://www.ultimatetennisstatistics.com/h2hHypotheticalMatchup?playerId1={p1_id}&playerId2={p2_id}'

response = session.get(url, headers={'User-Agent': 'Mozilla/5.0'})
doc = BeautifulSoup(response.text, 'html.parser')
win_prob_row = doc.find(string='Win Probability')
p1_win_prob = float(win_prob_row.parent.parent.find('h4').contents[0].replace('%', ''))
print(p1_win_prob)

Success! 🎉 Calling the API endpoints directly is now my preferred web scraping method. I experimented with several other sites to find the remaining numbers I needed, then I used this method to bring them into the project. Some endpoints returned JSON data, which can be parsed with the built-in "json" Python package rather than with Beautiful Soup.

After finishing this project, I found a tool called ScraperAPI that would have made life easier here. It allows scraping programs to bypass IP restrictions, retry automatically, execute JavaScript, and do many other things so that you don't need to scour the web to find a scrapable site like I did. They have a nice free plan to get started.

Configuring the Database

There are several reasons I needed to store data. First, the bet decision needs to be made before the match is played. I had to store it so that the program can check it against the result once the match is played to calculate the simulated payout. Second, the results need to be stored so that the total simulated payout can be calculated.

I chose PostgreSQL (Postgres) as the database since it is a free, open-source, relational database and is popular at the enterprise level. I could have used something simpler like Google Sheets or even Excel, but this was a perfect opportunity for me to get my hands dirty with SQL.

Originally, I wanted to host the database and the program online so that this could run around the clock, but I decided to keep everything running locally on my computer for simplicity. Running the program a few times a day would be sufficient to get the match odds, win predictions, and results, and I could do that manually to start. I installed Postgres locally as well as pgAdmin, a program where I could easily write SQL statements and manage the data. I set up a table named matches with the following columns, which evolved over time as I realized what I needed.

The next piece of the puzzle was connecting my Python script to the Postgres database. I needed to create SQL statements and send them to the database for execution. I found psycopg2, which is the most popular Postgres database adapter for Python, and installed it into my project. With this tool, I could now open a connection to my database, send it a string of SQL to execute, and close the connection when completed.

Crunching the Numbers

The Bet Prediction

Previously, the prediction we've referred to is the external prediction of who will win the match, which we scraped from Ultimate Tennis Stats. Now, the prediction we're referring to is the program's prediction of who to bet on - the Decision column in the database. In hindsight, my variable naming could have been more clear. 😅

Once the program scrapes what the upcoming matches are along with each player's betting odds, it can determine who to bet on. Here is the calculation of its prediction in Python. I've taken the betting odds scraped earlier and distilled them into p1 and p2 objects for each match, which have the properties name and odds for each player. p1_win_prob is the probability that we scraped earlier (61.4 in our example).

def get_factor(odds):
    if odds >= 0:
        return abs(odds)/100
    else:
        return 100/abs(odds)

def make_prediction(p1_win_prob, p1, p2):
    sim_p1_wins = p1_win_prob * 10
    sim_p2_wins = 1000 - sim_p1_wins

    p1_win = sim_p1_wins * get_factor(p1['odds'])
    p1_lose = sim_p2_wins * -1
    p2_win = sim_p2_wins * get_factor(p2['odds'])
    p2_lose = sim_p1_wins * -1

    p1_total = p1_win + p1_lose
    p2_total = p2_win + p2_lose

    global prediction

    if p1_total < 0 and p2_total < 0:
        prediction = 0
    else:
        if p1_total > p2_total:
            prediction = 1
        else:
            prediction = 2

    return [p1_total, p2_total, prediction]

The function get_factor determines how much money you win per dollar bet if that player wins. Then in make_prediction, it follows the same logic as shown earlier in the table.

1. Find the predicted number of wins for each player out of 1000 matches. In our example, it would be 614 for player 1 and 386 for player 2.

2. Find the total (net) payout after all wins and losses for a bet on each player

3. Bet on the player with the higher positive total. If neither is positive, don't bet.

It now has all the data it needs to upload its match prediction to the matches table as "Decision". First, it formats the relevant match data into a string, representing a single row insertion to the table.

def create_row_string(event):
    [p1, p2, start_epoch, match_id] = unpack_event(event)
    p1_win_prob = get_win_prob(p1, p2)
    [p1_total, p2_total, prediction] = make_prediction(p1_win_prob, p1, p2)

    row_string = f"""(DEFAULT, '{p1['name']}', '{p2['name']}', {p1_win_prob}, {p1['odds']}, {p2['odds']}, {p1_total}, {p2_total}, {prediction}, {start_epoch}, {match_id}, '{convert_time(start_epoch)}')"""
    return row_string

These strings are used to build an INSERT SQL statement to upload all of them into matches. The betresult column is excluded since the matches haven't happened and do not have results yet.

def insert_new_matches(match_data):
    start_string = f"""INSERT INTO {db_table} (Id, Player1Name, Player2Name, Player1Prob, Player1Odds, Player2Odds, Player1Total, Player2Total, Decision, StartEpoch, MatchId, DateTimeUTC) VALUES """
    value_string = ', '.join(list(map(create_row_string, match_data)))
    where_string = ' ON CONFLICT (MatchId) DO NOTHING'

    insert_string = start_string + value_string + where_string + ';'
    util.sql_command(cur, conn, insert_string)

The Simulated Payout

After the matches have been completed, we can come back and calculate the payout (betresult) based on a bet of $1 on each match. All we need for that is

1. The betting odds

2. Our bet decision from the previous step

3. The result of the match

The odds and decisions are already in our database, and we can scrape the match results from the web, so what's the best way to do this calculation for each match? There were two approaches I could take.

1. Pull the decision and odds out of the database, do the calculation in Python, and send the betresult back to the database.

2. Send the match result to the database first, do the calculation at the database level, and update the match rows with the betresult.

With option 1, I already had most of the logic for determining a betresult in Python, but I would have to make two SQL commands for each match: SELECT and UPDATE. With option 2, I would have to rewrite some of the logic as a function in SQL, but I would only need one SQL command per match: UPDATE. While option 1 may have been easier, option 2 seemed more efficient since the number of database calls was reduced.

First, I defined the SQL function get_bet_result. It takes the winner and loser of the match as parameters, pulls in the decision and odds from the table, and calculates the payout. PLpgSQL is a language built into Postgres that allows you to define functions and other logic to execute at the database level. I wrote out the function as a string in the Python script, and when the script runs it sends a CREATE command to the database to define it. The function is long so check out my repo if interested. Then, for each match, the program sends an UPDATE command that sets the betresult column to the returned value from get_bet_result(winner, loser) for the row in the table that correlates to the same match as the results.

def update_match(match_result):
    winner = util.sanitize(match_result[0])
    loser = util.sanitize(match_result[1])

    update_string = f"""
        UPDATE 
            {db_table} 
        SET 
            betresult = get_bet_result('{winner}', '{loser}')
        WHERE 
            (player1name LIKE '%' || '{winner}' || '%' OR player2name LIKE '%' || '{winner}' || '%')
            AND (player1name LIKE '%' || '{loser}' || '%' OR player2name LIKE '%' || '{loser}' || '%')
            AND CAST(EXTRACT(epoch FROM NOW()) AS BIGINT)*1000 - startepoch < 345600000;
    """

    util.sql_command(cur, conn, update_string)

Now that we can simulate payouts, we can get a sense of if this is a viable strategy! Let's open pgAdmin and check out our table.

Here is a close-up view of the first row.

In this match, Baez is the favorite against Darderi based on the odds and the win probability. Our program simulated a gain of $68.17 after 1000 $1 bets on him and a loss of $274.30 after the same bets on Darderi. Therefore, the decision is 1 (Baez, player 1). The betresult column is positive, meaning Baez won the match and we won our bet! We gained $0.41 on a hypothetical $1 bet.

Any match with a decision of 0 means we didn't bet on it, so the betresult is automatically 0. If match results aren't found, then betresult stays null. The odds data and match result data are coming from two different sources, so sometimes we don't get results. Oh well.

Running the Program

I can run the program manually now, but the ATP Tour doesn't wait for me. Matches are played all the time around the world, so I needed to automate running this script to gather the odds and results around the clock. I mentioned before that ideally this would run on a cloud server that is always alive, but for simplicity and cost, it's running locally on my computer.

I found a program called launchd, which allows Mac users to run programs automatically as long as the computer is awake. This is installed on Mac computers by default to run processes in the background. To add my own process, I needed to add a special .plist file to the /Library/LaunchAgents directory on my computer. Here's what it looks like.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>launchd_tennis_predictions</string>
  <key>RunAtLoad</key>
  <true/>
  <key>ProgramArguments</key>
  <array>
    <string>/usr/bin/python3</string>
    <string>/absolute/path/to/index.py</string>
  </array>
  <key>WorkingDirectory</key>
  <string>/absolute/path/to/working/directory</string>
  <key>StartInterval</key>
  <integer>3600</integer>
  <key>StandardOutPath</key>
  <string>/absolute/path/to/out.log</string>
  <key>StandardErrorPath</key>
  <string>/absolute/path/to/error.log</string>
</dict>
</plist>

The format is an HTML-like list of key-value pairs with the settings. All file and directory paths need to be absolute paths.

• Label - a name for the program

• RunAtLoad - whether or not to run when the computer boots up

• Program Arguments - an array of string arguments. Think of it as the arguments you would type in the command line. Instead of python3 index.py, it would be /usr/bin/python3 /absolute/path/to/index.py.

• Working directory - absolute path to the directory holding your program file

• Start Interval - the number of seconds between each execution

• Standard Out Path - absolute path to the file logging any output messages, such as print statements.

• Standard Error Path - absolute path to the file logging errors.

Now my file runs every hour as long as my computer is awake. I set up the scraping to pull odds for all available upcoming matches and match results for the last few days so that I have the best chance of gathering those while I use my computer normally for other things.

Testing

During development, I didn't want to run untested SQL commands on my table of real data, so I needed a test table. I created a matches_test database table and an index_test.py script file to act as my test environment. The script file checks its own file name and uses the corresponding database table. All development can be done in index_test.py and matches_test, then when I'm satisfied with the results I just copy index_test.py to index.py. It's not the most robust test environment but it works well enough for my needs.

The Results

The measure of success for my hypothesis is simply the sum of the betresult column, across all matches. Here is the SQL statement I used to get the sum, along with the total dollars bet and rate of return on our simulated investment.

SELECT 
    SUM(betresult) AS payout,
    COUNT(betresult) AS dollars_bet,
    SUM(betresult)*100/COUNT(betresult) AS rate_of_return
FROM 
    matches
WHERE
    betresult != 0 AND betresult IS NOT NULL;

I wrapped this in another Python script called get_result.py so that I could quickly check from the terminal rather than booting up pgAdmin. Drumroll, please!

After about 6 months, the program made predictions on 909 matches. Some match results were missed if my computer was off for a while, and some matches it chose not to bet on. After simulating $366 in bets, we have a loss of $48.60 for a rate of return of -13.28%. Darn!

The script showed that this method in particular would not make money, but I was successful in testing my idea for free! While this program did not make me rich, I gained a wealth of knowledge and had a blast while doing it.

The End

I hope you enjoyed this post! Please leave a like and a comment as I would love to hear your feedback. Was there a flaw in my logic? Could I have improved this somehow? Let me know!

How I got here, in a nutshell

Last year, I changed careers to software development. I previously worked in product management within the tech industry, designing new features for software products before handing them over to the development team to be built. I have always loved problem-solving, and I decided that I wanted to build the features myself. By this point in my career, I had dabbled with small coding projects and had definitely caught "the itch". However, with the pandemic and other life events going on, I couldn’t take the leap to pursue a career where I could code full-time.

After several years of pondering the decision and waiting for the stars to align, I quit my job so that I could gain the necessary skills at Carolina Code School, a full-time web development bootcamp. I am now a software engineer at SIOS Technology Corp., where I get to develop high availability and disaster recovery solutions for cloud systems. Sounds crazy when I think about where I was just last year!

What to expect

Since becoming employed, I have had more time to explore the wide world of web development (pun intended). People ask if I prefer frontend or backend development as if it's a binary decision, but I can't pick just between those as there are so many interesting topics out there. I am currently interested in frontend frameworks, full stack development, UX, AI, machine learning, and cloud computing to name a few.

The field is expanding faster than any one person can keep up with, which makes it exciting to be a part of. My goal with this blog is to share what I build, the problems I solve, and the lessons I learn in my software development career.

Interested?

• Subscribe to my newsletter

• Check out my website

• Connect with me on LinkedIn

• View my projects on GitHub

Thank you!