Learning Python

  • Finished readingIntroducing Python
  • Implement most of the exercises from Introducing Python into IPython Notebook
  • Do CoderByte challenge and Coding Kata
  • Implement BART real-time trip estimator
    • Create a graph of all of the BART stations
    • Figure out which real-time estimate (which direction) to use
  • Implement web scraper for (healthcare / software engineering) using scrapy
  • Implement Blackjack with command line input
  • Analyze healthcare dataset using Pandas

Software engineering goals 2015

Software Engineering

  • Finish notes on Design Pattern
  • Understand DDD and implement possible bits
  • Learn Ruby on Rails, then Go, and finally Python; know Java (understand syntax)
    • Simple to-do app for RoR
    • BART real-time calculator for Go
    • Analyze healthcare costs with Python
  • Use Typescript for personal project, then reservation-next
  • Implement a developer tool (ES Autocorrect) - understand AST

Notes on Python

Note: Some of the points made below are specific to Python 3

  • Math:
    • Exponentiation (2 ** 3 == 8)
    • 35 / 5 = 7.0 (division runs float by default)
    • 7.0 / 2 = 3.5
    • 7 / 2 = 3.5
    • 7 // 2 = 3 (takes the floor of the integer)
  • Equality:
    • Just "=="; no "===" (will throw an error)
  • Basic functions:
    • # single line comment
    • print() - equivalent to console.log
    • int() - typecast into integer
      • note: doesn't work like parseInt - will throw an error if you try to do something like int("hi5")
    • str() - typecast into a string (even works on lists)
    • list() - typecast into a list
    • range() - gives you a list from 0 up to but not including the number passed in
      • range(4,8) # [4,5,6,7]
      • range(3) # [0, 1, 2]
  • Booleans are True or False (must be capitalized)
    • None, 0, empty list / dicts / string are False. All others are True.
  • None is an object (is this like a null object?)
  • Strings are immutable; you cannot do implicit conversion between string and integers (unlike Javascript which does type coercion)
  • Lists are mutable; Tuples are immutable and take less space in memory
    • append() equivalent to push() in JS array
    • pop() removes and returns the last element 
    • a_list_of_numbers.sort(Reverse=True) (sort in memory)
    • sorted(a_list_of_numbers) - returns a new sorted list
    • Concise syntax for doing slices [start:end:step]
      • a_list[::] - shorthand for copying a list
    • You can do destructuring (aka unpacking) for list and tuples
    • list.extend() li.extend(li2) (adds elements of the second list into the first list)
  • Dictionaries are key-value in-memory stores (similar to Plain Old Javascript Object POJO)
    • Note: Unlike Javascript, there is no implicit string conversion on the keys which means you can use variables for the key
    • dict1.update() adds key-value pairs from the second dict to the first dict
  • Remove elements from list or dicts with del keyword (e.g. del li[2])
  • Functions have positional (which one came first) and keyword (what name is assigned to the argument) arguments 
    • Example of keyword arguments:
      • def func1(name, age):
      • func1(age=24, name="will")
    • def varArgs(*args):
    • def kwArgs(**kwargs):
    • def bothArgs(*args, **kwargs) (I believe the positional args globber must come before the keyword args globber)
    • Automatically uses local scope in function. To refer to global scope, must use global keyword
    • Lambdas / anonymous functions:
      • (lambda x: x > 2)
  • List comprehension is a way to create a list through another list
  • Module system: Module is a Python file (you don't need to specify the .py file extension)
    • Syntax for importing
      • import random - for importing an entire module
      • from math import ceil, floor - import specific functions from a module
      • import math as m - alias an import
  • Template string is """ and """
    • Allows you to do multi-line strings
    • Sometimes used for commenting
  • Conditionals:
    • if x > 10:
    • elif x > 0:
    • else:
  • For loop:
    • for dog in ["alfred","beagle"]:
      • print(dog)
  • Being Pythonic
    • Use snake_case for identifiers
    • Check for existence in list: (e.g. 3 in integer_list)


Sources:

  • Introducing Python: Modern Computing in Simple Packages
  • TDD Python
  • Django docs https://docs.djangoproject.com/en/1.8/intro/overview/

Notes on Refactoring to Patterns

  • Replace Constructors with Creation Methods (also known as Static Factory Method) - Use a static or nonstatic method to create instances. For example: TodoStore.createItem(data)
    • If you have too many Creation Methods, you can apply Extract Factory and create a factory. (e.g. if you have a Todo class with 5 methods each to create a different type of todo item, you create a Factory that holds all these creation methods so the intent of the original Todo class is clear).
  • Move Creation Knowledge to Factory When you have too many classes that unnecessarily know how to create instances of a class, you combine all those creational methods to a single Factory class
    • Creation Method - a static or nonstatic method returns an instance of a class
    • Factory - a class that implements one or more Creation Methods
    • Factory Method - "a nonstatic method that returns a base class or interface type and that is implemented in a hierarchy to enable polymorphic creation" (p 71). https://en.wikipedia.org/wiki/Factory_method_pattern
    • Abstract Factory - "an interface for creating families of related or dependent objects without specifying their concrete classes" (Design Patterns, 87). https://en.wikipedia.org/wiki/Abstract_factory_pattern#Pseudocode
      • Every Abstract Factory is a Factory but not conversely.
  • Encapsulate Composite with Builder Simplify the creation of a composite (e.g. a parent DOM node with a couple of children DOM nodes) by using a Builder to deal with the complex details.
    • If you have a class with many options, then you would need a lot of constructors to cover the combinations. So Instead, you use a Builder factory which allows you to set the options one by one. https://en.wikipedia.org/wiki/Builder_pattern
    • A Builder class will have one method to actually get the instance which is called as the last step to the Builder.
  • Compose Method Turn a long block / function definition into a series of well-named small-bite function calls.
  • Replace Conditional Logic with Strategy - Instead of using conditional logic to determine which algorithm to use - you delegate to a strategy instance.
    • Strategy Pattern has the program select the appropriate strategy at run time. Only use Strategy pattern when you are going to change the algorithms a lot. Javascript example at: http://www.dofactory.com/javascript/strategy-design-pattern
      • defines a family of algorithms,
      • encapsulates each algorithm, and
      • makes the algorithms interchangeable within that family.
  • Replace State-Altering Conditionals with State - Similar to Strategy but with a different intent. State is concerned with WHAT you can do and Strategy is considered with HOW you do something
    • Extract class to a State Class
    • Copy each method that modifies state in the host class to the new State class
    • Create subclass for each type of state
    • Copy each method for the subclass state and simplify the method from conditionals
    • Repeat for each state subclass and method that modifies state
  • Replace Conditional with Polymorphism
    • If your host class (the context) has subclasses, and the different subclasses matches the variations of algorithms then this is a good choice for refactoring (p. 130)
  • Replace Type Code with Subclasses - If the conditions to select the variations of the algorithm is based on type code, this is the preferred refactoring. 
  • Replace Conditional Dispatcher with Command - Command pattern separates who requests a command and who executes the command.
    • Use this when:
      • You need to dynamically modify the conditional logic for dispatching
      • You have to scroll a lot to understand the conditional & dispatching actions
  • Misc notes:
    • Abstract vs. concrete: abstract type is a type in a nominative type system which cannot be instantiated directly
    • Nominal vs. structural: nominal (Java) vs structural / duck typing (Typescript)
    • Template method pattern "is a behavioral design pattern that defines the program skeleton of analgorithm in a method, called template method, which defers some steps to subclasses" (source: Wikipedia).  first a class is created that provides the basic steps of an algorithm design. These steps are implemented using abstract methods. Later on, subclasses change the abstract methods to implement real actions. Thus the general algorithm is saved in one place but the concrete steps may be changed by the subclasses.
    • Algorithm is just a step-by-step of instructions
    • invariant (or standard), and which are variant (or customizable)
    • Composite pattern is constructing a tree-like structure of objects (e.g. a parent DOM node with two DOM child elements)

Creational Patterns: creating instances

Structural Patterns: class and object composition

Behavioral Patterns: communication between objects

Note: Quotes are from the book Refactoring to Patterns by Joshua Kerievsky.

Highly recommend this resource on Design Patterns: https://sourcemaking.com/design_patterns

Design patterns are a language weakness: http://www.norvig.com/design-patterns/design-patterns.pdf

Pattern hierarchy: http://stackoverflow.com/questions/3883692/strategy-pattern-vs-command-pattern

React libraries

  • Redux (1.0 RC) Flux inspired architecture: https://github.com/gaearon/redux
  • React Router (1.0 beta) Ember-inspired router that enables nested views, lazy loading, animations, and not breaking the back button: https://github.com/rackt/react-router
  • FormatJs react-intl Internationalization date, time, string utility library supported by Yahoo: https://github.com/yahoo/react-intl
  • Babel Transcompiles future version of JS like ECMAScript 2015 and 2016 into ES5 that you can use today in the browser. Officially used by React to transcompile from JSX to JS.
  • Jest (by Facebook) Helper library built on top of Jasmine testing framework that auto-mocks, detects specs, and helps test async code. Doesn't seem very popular and when I briefly tried using it, it runs quite slowly: https://github.com/facebook/jest
  • React Native Targets mobile platform (iOS and eventually Android) instead of the web: https://github.com/facebook/react-native
  • Webpack Module loader (seems like the most popular in the React community and works particularly well with React Router for lazy loading): http://webpack.github.io/

Solutions to hard parts of web apps

  • Managing UI state. Good solution through unidirectional data flow as popularized by React and Flux (although these concepts can be applied to other frameworks like Angular).
  • Offline-capable / Persistence. When a user goes offline, will it send updates after they reconnect? What if they close the browser and re-open it, will it persist the changes to local storage?
  • Good mobile performance & responsive design. The first part is making sure that even on a less powerful computer like a smart phone (particularly for phones like cheaper Androids) your web app has a reasonable load time. The second part is having a design that's responsive to various dimensions of the viewport.
  • Scaling your data store / database. When you first start out, you can use something as small as SQLite, and then you can use dedicated VMs to host a database like MySQL, Postgres, or MongoDB. Managing a very large dataset will usually require a special solution but things like AWS Aurora are making it more accessible to every development team.
  • Monitoring & health in production. Keeping your web apps running smoothly in production requires sophisticated tooling (sometimes homegrown or open-source projects) but more often using dedicated solutions like New Relic, Loggly, and so on.
  • A/B testing & measuring business metrics. I separate this from monitoring because for more monitoring is more about getting very timely information (e.g. within the last 10 minutes) of how fast your response times, and how many errors you have received. Measuring is more about looking at which parts of the product have the biggest impact on the business. The most obvious KPI is revenue - can you attribute a set of features to an improvement to revenue? Mixpanel seems like an increasingly popular choice for this.
  • Continuous integration & deployment. Few people will disagree these are good practices, yet they are practiced consistently in most places. Using an open-source tool like Jenkins, Go.cd, or Buildbot or a solution like CircleCI, Codeship, and so on can help address this. Dropbox is developing a very interesting solution on Mesos but it's still highly experimental (https://github.com/dropbox/changes)
  • Cross-browser compatibility.  Subtle things break amongst the four different browser vendors like a button that can't be clicked on. The reactive solution is to use a tool like Sauce Labs or Browser Stack to automate cross-browser and the proactive solution is to use a battle-tested framework like Bootstrap or Foundation. While those popular frameworks will deal with most issues, you will still probably need to do some sanity checking to make sure your site is OK across browsers.
  • I18n. Scaling an app from just US to a truly global product takes a lot of effort. First, you need to get translations done (risky to rely on automatic translations using a service even like Google's). Second, you need to properly detect locale and display the right strings of text to users. Finally, you need to QA that the translations really work, which is hard to QA well with an English-speaking staff.

Links on Facebook Relay

Videos

React.js Conf 2015 - Data fetching for React applications at Facebook by Jing Chen / Daniel Schafer. Introduction video to GraphQL and Relay.


Mutations and Subscriptions in Relay by Laney Kuenzel. A dive into how the Relay framework makes it simple for developers to implement mutations (data writes) and real-time updates. 

Note: I'm still waiting for JSConf US 2015 to actually release the video. The speaker notes are quite detailed but I think it would be a lot easier to understand with the actual talk :)


Joseph Savona - Relay: An Application Framework For React at react-europe 2015 Follow-up talk on Relay. 


Articles

https://facebook.github.io/react/blog/2015/03/19/building-the-facebook-news-feed-with-relay.html

Notes on React: Up and Running

Notes from React: Up and Running (I purchased the early release book, use WCYAZ coupon code to get 50% off):

  • React.DOM shows you a list of HTML elements that you can use as React components
    • Ex: React.DOM.h1(DOMAttributeObject, childComponent, e.g. a textString)
    • This is just another way of doing React.createElement("h1", DOMAttributeObj, childComponent)
  • Special DOM attributes
    • className - because 'class' is a reserved keyword in JS
    • htmlFor - because 'for' is a reserved keyword in JS
    • style - cannot pass string, use JS object
      • Properties must be camel cased (e.g. fontFamily instead of font-family)
  • React.render(component, DOM location)
  • Render is the only required method for a React component
    • Must return only a single DOM node
  • componentFactory = React.createFactory(Component); component = componentFactory()
  • Treat this.props as read-only; it should only be changed by having new properties passed in from parent component
  • Treat this.state as read-only; it should only be changed using this.setState()
  • this.propTypes allows you to do run-time validation to make sure you received the expected types for the properties passed in
    • ex: React.PropTypes.func.isRequired - see full list on React's docs
    • Shows a warning in console only during development mode
    • you can have additional properties in addition to propTypes (which is optional having at all)
  • DOMAttributeObject
    • <h1 className="{willStyles}" onChange="{this._changeHandler}">
  • React does event handling for you because doing it by hand has several issues:
    • Inconsistencies between browsers (hence the need for libraries like jQuery)
    • Event Delegation - to improve performance you listen to one parent element and then figure out which sub-element was clicked.
  • Lifecycle Methods
    • First-time loading:
      • componentWillMount(): before render()
      • componentDidMount(): after initial render() and node is on DOM
    • Updating:
      • componentWillUpdate(object nextProps, object nextState): before render() and changes are on the DOM
      • componentDidUpdate(object prevProps, object prevState): after render() and changes are on on the DOM
      • componentWillReceiveProps(object nextProps): if you need to do this.setState on receiving new props
      • shouldComponentUpdate(object nextProps, object nextState): for performance reasons (use immutable-js)
    • Removing:
      • componentWillUnmount(): before a component is unmounted from the DOM
      • componentDidUnmount(): after a component is unmounted from the DOM
  • Parent-child component lifecycles:
    • Parent will mount or update
    • Child will mount or update
    • Child mounted or updated
    • Parent mounted or updated
  • Mixins are a way of extending methods from one object to another in JS
    • example: mixins: [mixin1, mixin2]
  • JSX is just syntax sugar. Using '{' and '}' inside of the HTML-like syntax allows you to use Javascript. For example: <a className="{jsObjWithStyles}">text</a>
    • Must close every tag (e.g. <br />
  • Best way of commenting JSX: { /** **/ }
  • HTML to JSX (https://facebook.github.io/react/html-jsx.html)
  • React automatically escapes HTML tags
  • Spread operator for React reduces boilerplate:
    • var attributes = {href="google.com", styles={font: 14px} }
    • Instead of: render( return <a href={attributes.href} styles={attributes.styles}>)
    • Just do: render( return <a {...attributes}>)

    Redux from scratch

    In my post earlier today, I posted resources regarding this new flux-inspired library called Redux. After playing around with their two examples, I think I have a good idea of how it works now and I'd like to basically re-explain the library in my own words for two reasons: 1) it will help me solidify my understanding and 2) the docs for Redux is pretty good but is still a work-in-progress.

    Redux essentially gently nudges you to a more a functional way of doing Flux-esque architecture which enables neat developer tools like their hot-loading / time machine as discussed in the video. Many more benefits are outlined by the author here.

    State is in Dispatchers, Not Stores

    The first place to start is to understand that all the state is now managed by the dispatcher and not the individual stores. The stores instead become essentially a collection of functions to update the state.

    The app bootstraps from this:

    Top-level Components get State from the Store

    https://github.com/gaearon/redux#smart-components

    I'm calling it top-level components to be more descriptive but technically it doesn't have to be top-level. You can also just think of these as the "stateful components".

    So what's happening here is that the Connector is a parent component that's basically picking which state to get. Then, in the renderChild, the first parameter is the state that you have selected, and the second is a callback function for the dispatcher. You then do bindActionCreators, which binds the action creators to the dispatcher. You can see how it's implemented in the bindActionCreators.js file which is only 7 lines.

    export default class TodoApp {
      render() {
    return (
    <Connector select={state => ({ todos: state.todos })}>
    {this.renderChild}
    </Connector>
    );
    }

    renderChild({ todos, dispatch }) {
    const actions = bindActionCreators(TodoActions, dispatch);
    return (
    <div>
    <Header addTodo={actions.addTodo} />
    <MainSection todos={todos} actions={actions} />
    </div>
    );
    }
    }
    So where does this state.todos get defined? It's in the stores!

    Stores set initial state and then take in state from then on

    Below is a store from the counter example in the repo. Here you are changing the state. This is where you can use immutable.js so you are not mutating the current state object.

    import { INCREMENT_COUNTER, DECREMENT_COUNTER, INCREMENT_COUNTER_BY_THREE } from '../constants/ActionTypes';

    export default function counter(state = 0, action) {
    switch (action.type) {
    case INCREMENT_COUNTER:
    return state + 1;
    case INCREMENT_COUNTER_BY_THREE:
    return state + 3;
    case DECREMENT_COUNTER:
    return state - 1;
    default:
    return state;
    }
    }

    Child (state-less) components hook to actions

    I've pasted below a simplified version of the TodoItem from the TodoMVC example. Here, I've created an action to move a to do item to the top of the list, if you click on the "Move Up" link. The link triggers the moveTodoUp action which is passed into it from its parent component (and originates from the Top-level component where we did the bindActionCreators.

    export default class TodoItem extends Component {
      static propTypes = {
    moveTodoUp: PropTypes.func.isRequired
    };

    render() {
    const {todo, markTodo, deleteTodo, editTodo, moveTodoUp} = this.props;
    }

    return (
    {element} <a style={willStyles} onClick={()=>moveTodoUp(todo.id)}>Move up</a>
    );
    }
    }

    A simple action

    The moveTodoUp action that you saw earlier is defined here in the ActionCreators file. It's pretty simple, all it does is return a type and what it was passed in.

    export function moveTodoUp(id) {
      console.log("moveTodoUpId", id)
    return {
    type: types.MOVE_TODO_UP,
    id
    };
    }

    Conclusion

    Overall, it was fun learning this library. I think it has some interesting concepts and the learning curve wasn't too bad. I think the docs can be organized a little better and have a few more examples (the TodoMVC and counter examples were great, so maybe including snippets from those in the docs itself). The one suggestion I would have is to see whether you can use the ES6 global symbol registry instead of having a separate file to write down constants for action strings.