OOPSLA 2009 Wednesday, October 28th

Jeannette Wing, CMU, Frontiers in Research and Education in Computing

Jeanette said that there has been a paradigm shift.

Not just about computing's metal tools (transistors and wires), but also our mental tools (abstraction and methods)

Is this really a new paradigm shift? Maybe for the National Science Foundation (NSF), certainly not for anyone that has been working with software the last ten years.

The limits of Moore's Law forces the NSF to focus on programming languages and abstractions.

Three drivers of computing research, Society, Science, Technology.

Encouraged research areas:

• Data Intensive Computing
• Cloud Computing

• Map-Reduce

• Cyber-Physical Systems (computational core that interacts with the physical world)

• Smart vehicles
• Smart Flyers

• Smart Devices

• Network Science and Engineering

• Understand the complexity of large scale networks
  • Software Engineering for the Next Internet

• Trustworthy Computing

• Socially Intelligent Computing

• Humans are still much better at image recognition

• Programs where the human is a port of the program.

• IT and Sustainability (Energy, Environment, Climate)

• Computer Science and Economics

• AdSense

• eBay

• Computer Science and Biology

Agile development: Overcoming a short-term focus in implementing best practices, Karthik Dinakar

In the project they concluded on the following good practices.

• Good version control
• Coding guidelines
• Build automation
• Unit testing framework
• Automatic sanity tests
• Accurate task estimations
• Effective pre-spring planning
• Solid design discussions
• Involve QA and Operations
• Effective post-sprint-reviews

They also thought that it is difficult to implement best practices under way.

They had many problems:

• Integration was not in the plan
• Sprint backlog changing all the time.
• Long sprint meetings

Management did not allow them to implement the changes that they needed and the reason was "It's Agile!"

All in all, their problems seemed to be the usual, they had no idea of what it meant to do Scrum in the first place. Don't people read books anymore?

Are systems Green? Panel with Steve Easterbrook et al

• A Google query has a Carbon footprint.
• We don't know what it is.
• It may actually be less than the energy it uses since it may permit the person posing the query to save a lot of energy.
• The term Green is just marketing bullshit.
• It is not measurable at all.
• Carbon Emissions are permanent.
• It won't go away even if we stop burning any carbon today.
• The goal of emissions are ZERO, anything else is not sustainable.

There is a book called Green IT for Dummies

• What can we as a computing industry do.
• Analyze the problem.
• Make a list of what we need and can do.
• Create a wiki.

If you are going to read an article for more than three minutes, YOU SHOULD PRINT IT OUT! Believe it or not, but check your facts.

Architecture in an Agile World, Panel with Steven Fraser et al

Randy Miller: We allow the customers to make changes, but we don't tell them what the cost will be. If we have no architecture the cost will be high. The people behind the agile manifesto were all good at architecture and that is why the importance of architecture was not emphasized in the manifest.

Bill Opdyke: Some people seem to see a difference between architects and agilists. But most people that are good don't have the this problem. Architects can learn from agilists that change is not that hard. Agilists can learn from architects that architecture matters. There is a middle ground.

Ethan Hadar: We need a stable vision of what the architecture should be while delivering solutions iteratively. Show me the architecture road map of your product, because it will be integrated with another program in nine months. Accountability and responsibility for the architecture is important.

Dennis Mancl: You need to plan architecture early, or you will have to do it later on and then it will be harder.

Audience: How does an architect work in an agile team?

Randy Miller: From an agile team, you're all part of the team. There are no distinctions. The architect is in the team to make sure that

The architecture in the system is the point in time in which you have to step back and think about how everything interacts.

Ethan Hadar: The architect is the person, who needs to interact with the testers and operations to explain how and why the system is the way it is.

Dennis Mancl: The architect is responsible to the stake holders and he is responsible to know the problem domain.

Irit Hadar: The architect's role is to take a step back and say when it is time to review the architecture.

Audience: The architecture is what you get, regardless of what you do.

An interesting discussion, that could not find a consensus to if there is a need for a single architect or not.

Agile Anthropology and Alexander's Architecture, Jenny Quillien, Dave West, Pam Rostal

Do we need to pay any interest of to Christopher Alexanders' new book The Nature of Order?

The software community has always taken interest in Alexanders' books, but we take interest in the wrong things. We have only understood some rules, we have not grasped the deeper part of it, because we don't understand the culture of architecture.

In a pattern language, we looked at patterns, but we dismissed the QWAN. We missed the holistic point-of-view. Everything is part of the system, people, organizations. He is writing about things that are multi-dimensional and multi-faceted, not things that are simple and exact.

The Nature of Order contains the same multi-faceted ideas and if we look at it with the same eyes, we will miss the point again.

Alexander looked at centers and centers affect other centers. And there is no right or wrong, there are only degrees.

In Alexanders' world there is only one system, the Universe. Everything is connected!

Writing Code for Other People, Tom Mullen

Chunking and Memory

• The mind groups memory into chunks. Most chunks are stored in long-term memory. Out conscious is in long-term memory.

Short-time memory can only hold about seven relations. Short-time memory is also short:) This gives us a time-limit when traversing code.

Meyers' open-closed principle is an echo of the mind's way to learn things.

If the code is a reflection of our brains, then most brains contain spaghetti.

Analogies

Analogies is the mapping from one thing to another.

Conclusion

Our brains are not good a processing more than 4 chunks at the time. This implies that we should write methods with less than four lines. Classes with less than four methods, modules with less than four classes, and applications with less than four modules.

OOPSLA 2009 Tuesday, October 27th

Barbara Liskov, the Power of Abstraction

OOPSLA 2009 opened with Barbara Liskov as the keynote speaker. She is famous for, among others, the Liskov Substitution Principle. This principle states that:

A subtype should be substitutable for its super-type.

That is, it should be possible to use a subtype in the same way as if the type itself was used. Any difference in behavior should NOT be noticeable to the client.

A History of Abstract Data Types

Data abstraction was developed as a solution to the software crisis. A crises that is just as present today as it was then. This was in 1968, the time when Dijkstra wrote the paper Go To Statement Considered Harmful. The problem with gotos is that it is difficult to know the context in which a statement is used.

Other important papers at the time were. Nicholaus Wirth's paper Program Development by Stepwise Refinement on 1971, about top-down design and David Parnas' paper Information Distribution Aspect of Design Methodology in which he stated.

The connections between the modules are the assumptions which the modules make about each other.

These assumptions are often much more than the simple interfaces that we see today. It includes the whole context in which the module is used.

Barbara then wrote a paper called A Design Methodology for Reliable Systems. The ideas from this paper was later reused in the context of programming. When seen from the outside it is apparent that the same technique she used when creating the Venus Operating System, Partition State, could be used when building programs but it was not at the time. Where do ideas come from? Perhaps the time is just right.

Other influential papers that are still valid today are: Hierarchical Program Structures by Dahl and Hoare and Protection in Programming Languages by Morris, introduced the early ideas of encapsulation and Global Variable Considered Harmful by Shaw and Wulf.

In 1973, the paper on Abstract Data Types was published and Liskov the realized her ideas in CLU. Its worth noting that CLU was way ahead of its time. It included features, like data encapsulation, exceptions, iterators via yield but no inheritance. She doesn't think that inheritance is very important and that it complicates things.

The Liskov Substitution Principle didn't appear until 1983, when she held a speech here at OOSLA, when she had noticed that inheritance was used for two different things that was not very well understood. It is used for:

• Implementation inheritance, which violates encapsulation.
• Type Hierarchy, and this was not very well understood.

She ended up with noting that modularity based on abstraction is the way things are done now. It wasn't at the time.

She also pointed some challenges that still exist:

• New abstraction mechanisms
• Massive Parallel Computers
• MapReduce?
• Transactional Memory?
• Internet Computer
• Storage and computation
• Semantics, reliability, availability, security

And she also made the point that

Readable programs are much more important than writable programs.

When the questions were opened, it was interesting to note that among the questioners were Phil Wadler (Haskell), Andrew Black (Traits), Guy Steele (Scheme, Fortress), Dave Ungar (Self) and Ralph Johnsson (GoF).

Flapjax, a Programming Language for Ajax Applications

After Liskov's keynote I watched a presentation of a research paper about Flapjax, a language designed for web applications. It is based on event streams and the language itself is reactive. Flapjax is a Javascript-based language that can be used as a library.

The language introduces two new concepts Behaviors and Event Streams.

A behavior is a value that changes over time. It can be created like this.

// A variable that changes over time, every 100ms.
var nowB = timerB(100);

What is interesting is that the behavior is composable with normal Javascript functions. This is done by compiling or transforming the Javascript into Flapjax code.

If an expression is a behavior, all expressions whose values depend on it also become behaviors.

var nowB = timerB(1000); 
var startTm = nowB.valueNow(); 
var clickTmsB = $E("reset", "click").snapshotE(nowB).startsWith(startTm); 
var elapsedB = nowB - clickTmsB;
insertValueB(elapsedB, "curTime", "innerHTML");

Programming with event streams is a little different from programming with behaviors. A behavior masquerades as an ordinary JavaScript object whose content just happens to change automatically. In contrast, an event stream is a new kind of value, with new primitives for programming over it.

Event streams and behaviors offer complementary views of the world. It is easy, however, to overstate their differences. Given an initial value, every event stream can be converted into a behavior: the behavior always has the value of the last event to have arrived on the stream, starting with the specified initial value until the first event arrives. Likewise, every behavior can be converted into an event stream: when the behavior's value changes, send the new value as an event.

Thomas W. Malone, Keynote Onward, the Future of Collective Intelligence

MIT Center for Collective Intelligence

Collective intelligence - Groups of individuals doing things collectively that seem intelligent.

Collective Stupidity is also very much existent.

New examples of collectively intelligence are: Google, the Web, Wikipedia, Linux, Digg, YouTube, etc.

How can people and computers be connected so that collectively they act more intelligently than any person, or computer?

Thomas showed a video of a crowd of people flying an airplane by turning a reflective shield green or red.

What are the genomes of collective intelligence?

Every activity has to answer four questions, Who? What? How? Why?

• There are two Who?-genes, crowd and hierarchical.
• There are three Why?-genes, money, glory and love.
• There are two What?-genes, create and decide.
• There are four How?-genes, collection (contest), collaboration, group decision (voting, consensus, averaging, prediction markets) and individual decision (market, social network).

Failure to get motivational factors (thw why?) right is probably the single greatest cause of failure in collective intelligence experiments.

Interesting examples are: Climate Collaboratorium, TopCoder, Kasparov vs. the World, Amazon Mechanical Turk and TurKit

What's coming?

Human Brain is very much like Global Network.

• We have global moods?

Quotes from We are the Web, Wired 2005

There is only one time in the history of each planet when its inhabitants first wire up its innumerable parts to make one large Machine.

Three thousand years from now, when keen minds review the past, this will be recognized as the largest, most complex, and most surprising event on the planet.

The Machine provided a new way of thinking (perfect search, total recall) and a new mind for an old species. It was the Beginning.

Brion Vibber, Wikipedia, Making your people run as smoothly as your site

As the number of people involved in a project grows, key decision-makers often become bottlenecks, and community structure needs to change or a project can become stalled despite the best intentions of all participants.

Instead of having a single admin looking at a page to decide if it is garbage, a group can vote if they think it is garbage. This allows the admin to delete pages without checking them first if everyone votes for deletion.

• People have limited time and patience.
• Waiting on other people is slow.
• People want to do what interests them, not deal with process!

Get out of peoples' way and let them do stuff!

Onward!

The Commenting Practice of Open Source, Oliver Arafat

An analysis of 80GB of Open Source code. The average code density is one comment per five lines of code or 19%.

Average comment density is independent of code size.

Strong variation by programming languages.

• Java code has an average of 26%.
• Perl code has an average of 11%.

Successful open source projects follow consistent comment practices.

Comment Density by Commit Size * Smaller commits have higher comment density.

Polymorphic System Architecture, Jeffery E Bryson

Run-Time polymorphism (RTP) has been used in the software community for two decades to satisfy dynamic reconfiguration, plug-n-play, extensibility, and system redundancy requirements. RTP is also used to construct software systems of systems. System engineers now have the same requirements applied to large-scale system architecture.

A Polymorphic System Architecture (PSA) uses the same technology, by applying it to the system architecture. By defining specific polymorphic relationship within the system architecture the system architect can reduce the system complexity and satisfy functional requirement.

Polymorphism reduces the code size, but it also reduces understandability.

Value Added

• Extendable/Reusable System Designs
• Dynamic reconfiguration
• An architecture that matures over time instead of becoming absolute.
• OO and Refactoring.

Conclusion

All in all, this was a good day with the keynotes being the highlights. The Onward sessions were interesting, but most of them, were of very little use to me.

Some thoughts from "A Theory of Fun"

I just finished reading the book A Theory of Fun by Raph Koster. It is funny how everything comes together once you start focusing and noticing certain patterns. It is a good book and worth reading even if you're not into game design.

I already learned from personal experience and from other books, that our conscious mind is terrible at multitasking. It is however very good at internalizing things, learning things so that the brain can perform them unconsciously, without conscious supervision. Raph calls this chunking.

The act of learning is about turning many steps into chunks that we don't need to think about as separate entities.

When we don't see something, we don't perceive it, but once we become aware of a certain pattern, we see it everywhere. Koster calls this noise.

Noise is any pattern that I don't understand.

A good game keeps us on the edge of our abilities constantly and once we learn something the game will become harder. This is a variant of flow. But since a game cannot continue for ever it is doomed to become boring once we have mastered it.

The destiny of games is to become boring, fun is the process and routine is its destination.

Koster also mentions some of his grandfathers carpentry practices.

• Work Hard on Craft
• Measure twice, cut once.
• Feel the grain, work with it not aginst it.
• Create something unexpected, but faithful to the source from which it sprang.

That is not bad advice for anything.

Javascript, the Esperanto of the Web.

I just gave the tutorial called Javascript, the programming language of the web at OOPSLA, this coming Sunday. It used to be called "the Esperanto of the Web", but no one seemed to know what that was, so I had to rename it.

If you want to learn good Javascript, there are three books that you need to read. Javascript, the Good Parts, by Douglas Crockford is a really good, really thin book that teaches you all that is worth knowing about the language. The other two books, the Little Schemer, and the Seasoned Schemer, by Matthias Felleisen and Daniel P. Friedman will teach you functional programming using Scheme. The reasons the Schemer books are so good for learning Javascript is that Javascript is more like a dialect of Scheme than it is like a dialect of C.

After reading these books you have a whole new appreciation for Javascript.

Here is one of the most beautiful functions in computer science, the Y-combinator in Javascript.

// The Y Combinator
var Y=function (gen) {
 return function(f) {return f(f)}(
  function(f) {
   return gen(function() {return f(f).apply(null, arguments)})})}

The fact that the Y-combinator can be written i Javascript shows the power and elegance of the language.

Lists in Scala

As with most functional languages, lists play a big roll in Scala. Lists contains, among others, the following operations.

// List in Scala or homogenous, they are declared as List[T]
val names: List[String] = List("Arnold", "George", "Obama")

// Lists are constructed from two building blocks :: and Nil
assert(names == "Arnold" :: "George" :: "Obama" :: Nil)

// Gets the first element of a list
assert(names.head == "Arnold")

// Gets the rest of the list
assert(names.tail == "George" :: "Obama" :: Nil)

// Checks if the list is empty
assert(List().isEmpty)

Instead of using head and tail, pattern matching is commonly used.

def length(xs: List[T]): Int = xs match {
 case Nil => 0
 case x :: xs1 => 1 + length(xs1)
}

From these simple functions a flora of functions is built.

// List length
assert(names.length == 3)

// ::: appends two lists
val namesTwice = names ::: names
assert(namesTwice == List("Arnold", "George", "Obama", "Arnold", "George", "Obama"))

// last gets the last element
assert(names.last == "Obama")

// init gets all but the last
assert(names.init == "Arnold" :: "George" :: Nil)

// reverse reverses the list
assert(names.reverse == "Obama" :: "George" :: "Arnold" :: Nil)

// drop drops the first n items
assert(names.drop(2) == "Obama" :: Nil)

// take keeps the first n items
assert(names.take(1) == "Arnold" :: Nil)

// splitAt, does both take and drop at the same time, returning a tuple
assert(names.splitAt(1) == (List("Arnold"), List("George", "Obama")))

// indeces, gives my the indeces of the list
assert(names.indices == List(0, 1, 2))

// zip, zips two lists together
assert(names.zip(names.indices) == List(("Arnold", 0), ("George", 1), ("Obama", 2)))

// toString returns a list as String
assert(names.toString == "List(Arnold, George, Obama)")

// mkString, lets you join the string with a separator
assert(names.mkString("-") == "Arnold-George-Obama")

There are also a few functions for converting to and from lists.

val array = Array("Arnold", "George", "Obama")

// Convert the list to an Array
assert(names.toArray == array) // Equality does not work for arrays
java.lang.AssertionError: assertion failed
 at scala.Predef$.assert(Predef.scala:87)
...

// If we convert it back it works
assert(names.toArray.toList == names)

// We can also mutate the array with copyToArray
List("Hilary").copyToArray(array, 1)
assert(array.toList == List("Arnold", "Hilary", "Obama")) 

// elements will give me an iterator
val it = names.elements
assert (it.next == "Arnold")

Pretty slick, but now it is time for the good stuff, Higher Order Functions!

// map, converts from one list to another, notice the placeholder syntax (_)
assert(names.map(_.length) == List(6 ,6, 5))

// Get the first char of the words
assert(names.map(_.charAt(0)) == List('A', 'G', 'O'))

// Get the names as lists
assert(names.map(_.toList) == List(List('A', 'r', 'n', 'o', 'l', 'd'),
 List('G', 'e', 'o', 'r', 'g', 'e'), List('O', 'b', 'a', 'm', 'a')))

// When you have a list of lists, you can use flatMap
assert(names.flatMap(_.toList) == List('A', 'r', 'n', 'o', 'l', 'd',
 'G', 'e', 'o', 'r', 'g', 'e', 'O', 'b', 'a', 'm', 'a'))

// Filter is used to filter out specific elements that satisfy the predicate.

// Filter out all names of length 6
assert(names.filter(_.length == 6) == List("Arnold", "George"))

val chars = names.flatMap(_.toList)

// Filter out all chars larger than 'a' (capitals are smaller in ascii)
assert(chars.filter(_ > 'a') == List('r', 'n', 'o', 'l', 'd', 'e', 'o', 
'r', 'g', 'e', 'b', 'm'))

// And combine them
// Give me the first letter of all words with length 6
assert(names.filter(_.length == 6).map(_.charAt(0)) == List('A', 'G'))

There is a bunch of other useful functions based on filter.

// partition returns a pair of list (satisfied, not satisfied)
assert(names.partition(_.length == 6) == (List("Arnold", "George"), List("Obama")))

// find returns the first element that satisfy the predicate
// Since this function may not be satisfied, an optional value is used
assert(names.find(_.length == 6) == Some("Arnold"))

// An optional value returns Some(value) or None
assert(names.find(_.length == 7) == None)

// takeWhile and dropWhile take resp. drop while the predicate is fulfilled
assert(chars.takeWhile(_ != 'o') == List('A', 'r', 'n'))
assert(chars.dropWhile(_ != 'm') == List('m', 'a'))

// Span does both at the same time
assert(chars.span(_ != 'o') == (List('A', 'r', 'n'), 
List('o', 'l', 'd', 'G', 'e', 'o', 'r', 'g', 'e', 'O', 'b', 'a', 'm', 'a')))

// forall checks that a predicate is true for all elements of the list
assert(!chars.forall(_ == 'a'))
assert(chars.forall(_ >= 'A'))

// exists checks that a predicate is true for some element of the list
assert(names.exists(_.length == 5))
assert(!names.exists(_.length == 7))

// sort, sorts a list according to an ordinal function
assert(List(3, 7, 5).sort(_ > _) == List(7, 5, 3))

The fold functions, fold left (/:) and fold right (:\) inserts operators between all the elements of a list. The difference between them is whether they start or end with the base element.

fold left: (0 /: List(1, 2, 3)) (op) = op(op(op(0, 1), 2), 3)

fold right: (List(1, 2, 3) :\ 0) (op) = op(1, op(2, op(3, 0)))

// Define the sum function for lists with fold left
def sum(xs:List[Int]): Int = (0 /: xs)(_ + _)
assert(sum(List(2, 3, 4)) == 9) 

// Define the product function for lists with fold right
def prod(xs:List[Int]): Int = (xs :\ 1)(_ * _)
assert(prod(List(2, 3, 4)) == 24) 

// Define reverse in terms of fold
def reverse[T](xs: List[T]) = (List[T]() /: xs) ((ys, y) => y :: ys)
assert(reverse(List(1, 2, 3)) == List(3, 2, 1))

Thats it for the methods of the List class. In the Companion List object we also find some useful functions. We have been using one of them, all the time.

List.apply or List() creates a list from its arguments.

Apart from this one, there are some other worth mentioning.

// List.range creates a list of numbers
assert(List.range(1, 4) == List(1, 2, 3))
assert(List.range(1, 9, 3) == List(1, 4, 7))
assert(List.range(9, 1, -3) == List(9, 6, 3))

// List.make, creates lists containing the same element
assert(List.make(3, 1) == List(1, 1, 1))
assert(List.make(3, 'a') == List('a', 'a', 'a'))

// List.unzip zips up a list of tuples
assert(List.unzip(List(('a', 1), ('b', 2))) == (List('a', 'b'), List(1, 2)))

// List.flatten flattens a list of lists
assert(List.flatten(List(List(1, 2), List(3, 4), List(5, 6))) == List(1, 2, 3, 4, 5, 6))

// List.concat concatenates a bunch of lists
assert(List.concat(List(1, 2), List(3, 4), List(5, 6)) == List(1, 2, 3, 4, 5, 6))

// List.map2 maps two lists 
assert(List.map2(List.range(1, 999999), List('a', 'b'))((_, _)) == List((1, 'a'), (2, 'b')))

// List.forall2
assert(List.forall2(List("abc", "de"), List(3, 2)) (_.length == _))

// List.exists2
assert(List.exists2(List("abc", "de"), List(3, 4)) (_.length != _))

And as if all this was not enough, Scala also support For Expressions. In other languages they are commonly known as List Comprehensions.

A basic for expression looks like this

for ( seq ) yield expr where seq is a semicolon separated sequence of generators, definitions and filters.

// Do nothing
assert(names == (for (name <- names) yield name))

// Map
assert(List('A', 'G', 'O') == (for (name <- names) yield name.charAt(0)))

// Filter
assert(List("Obama") == (for (name <- names if name.length == 5) yield name))

val cartesian = for (x <- List(1, 2); y <- List("one", "two")) yield (x, y)
assert(cartesian == List((1, "one"), (1, "two"), (2, "one"),  (2, "two")))

// And now the grand finale, the cartesian product of a list of list
def cart[T](listOfLists: List[List[T]]): List[List[T]] = listOfLists match {
 case Nil => List(List())
 case xs :: xss => for (y <- xs; ys <- cart(xss)) yield y :: ys
}
val cp = cart(List(List(1,2), List(3,4), List(5,6))) 
assert(cp == 
  List(List(1, 3, 5), List(1, 3, 6), List(1, 4, 5), 
  List(1, 4, 6), List(2, 3, 5), List(2, 3, 6), 
  List(2, 4, 5), List(2, 4, 6)))

Ain't it beautiful so say!

Scream, Project Management for the Real World

Many companies today find themselves in a situation where they have a working product, but adding new features takes forever. Even worse, when new features are added, old features stop working.

To solve this problem many companies have adapted Scrum. Scrum has a nice lightweight appeal. All you need is:

• A product owner who cares for a backlog with prioritized stories.
• A team that cares about their craft and take responsibility to deliver a subset of new features every month.
• A scrum master that makes sure the product manager and the team are playing by the rules of Scrum.

Thats it, the recipe for success...

But, what if you are not living in la-la-land where everyone on the project cares about the product?

What if your product manager doesn't keep a prioritized log of testable stories because she doesn't care. She just works here!

What if your team cares more about going surfing, than delivering well-tested, high-quality code.

Enter Scream!

Scream

Scream is project management for the real world! Scream is the way of managing unmotivated development organizations. The method itself is not new, it has been used for centuries to manage everything from husbands to entire countries.

In Scream, all you need is:

• A product manager who handle the requirements.
• A team whom will develop the requirements.
• A Scream master who will make sure that the team is playing by the rules of Scream.

At first glance, it looks deceptively like Scrum, but the rules are different: The Scream master is responsible for the product being delivered at high quality and may use any means he sees fit to make it work.

This makes all the difference in the world.

Now all you need is a Scream master with enough gust to deliver.

The Scream Master

The ideal Scream master is Begbie in Trainspotting. He has all the characteristics of a good Scream master:

Francis Begbie is an aggressive pit bull terrier, a monstrous, brawling hard man ready to explode at any moment, at anyone, for any reason. Begbie isn't afraid to test his fighting prowess against the largest of opponents. "Begbie didn't do drugs, he did people," says Renton. His sole ambition seems to be to jack someone in.

The Process

After you have selected the Scream master, you have to let him know that he will be judged on the performance of the entire development team, including, the product owner.

You also need to set up some acceptance criteria for what done is:

• All stories in the backlog, must be SMART, Specific, Measurable, Attainable, Realistic and Timely.
• The code should be DRY.
• 100% unit-test coverage of all non-trivial methods.
• Acceptance test for all stories.

The you set the project in motion.

Some Typical Scenarios

The backlog items are not SMART.

The product owner says: "I didn't have the time." Begbie: Slaps her face, "You daft c**t, these items better by SMART, right f***in' now, or I will glass you."

Bugs appear in production, due to missing unit tests.

A developer says: "It worked on my machine." Begbie: Punches him in the nose, "You f***in' buftie, if one more bug enters the system on your account you've f***in' had it."

Typical stand-up meeting:

You ken me, I'm not the type of c**t that goes looking for f***in' bother, like, but at the end of the day I'm the c**t with a pool cue and you can get the fat end in you face any time you f***ing want, like.

A scream is not only against his team and product owner, he is against everyone. This makes him excellent for dealing with impediments. Project management is all about communication and motivation and no one can get the message through like Begbie.

Begbie: "I need access to the Active Directory." SA: "I don't have the time." Begbie: "I need access to the Active Directory." SA: "You need to fill out this form." Begbie: "I NEED ACCESS TO THE ACTIVE DIRECTORY." SA: "OK, here you go."

Notes on Greg Young's on DDD

I watched Greg Young talk about DDD on InfoQ. Here are my notes on the talk.

• Only use domain driven design on appropriate projects. Most projects are not suitable.
• Use state transition event streams to communicate between different bounded contexts.
• Bounded contexts are one of the very keys of domain driven design. The same word may have different meanings in different contexts, and this is OK.
• Use OO, avoid setters. Objects have behaviors, not shapes.
• If you always have valid objects, you avoid the problem of having to check if an object is valid all the time. IsValid is not the solution.
• Always use the domain experts and end-users language.
• Separating commands from queries gives the benefit of eventually consistent, queries may read from a different place than the commands.
• Coupling is not a problem, if it's in the same layer.
• Model the view, such as screens, as reports, with no transactional behavior.
• Explicit state transitions remove the need for auditing. They are the audit.

Git undo, reset or revert?

If you have found this page you probably came here since you wanted to clear your working directory from all the changes that you have made.

The simple answer is:

# Clear working directory tree from all changes
$ git checkout -f HEAD

This is, however, not the best way to do it. A better way is:

# Clears the working directory tree, and stashes all the changes.
$ git stash

git stash allows you to get your changes back any time you need them in case you change your mind. It is also possible to inspect and manipulate the stashes.

# List all the stashes
$ git stash list
stash@{0}: WIP on admin_ui: 0c1a80a Removed annotation from JdbcAdminService, it is now explicity initialized in the applicationContext.
stash@{1}: WIP on admin_ui: 14e12e6 Added foreign keys for UserRole
stash@{2}: WIP on master: d188ecd Merge branch 'master' of semc-git:customercare
stash@{3}: WIP on master: 3763795 More work on user_details.
...

# Apply the latest stash, and remove it from the stack
$ git stash pop

# Apply a named patch, but leave it on the stack
$ git stash apply stash@{2} 

# Drop a stash
$ git stash drop stash@{3} 

# Clear the entire stash stack (almost never needed)
$ git stash clear

# A better way to purge the stash
$ git reflog expire --expire=30.days refs/stash

What about git reset then, it sounds like it should do about the same as git co -f HEAD. It doesn't. git reset is used for setting the current reference pointer, HEAD.

# Reset the latest commit, and leave the changes in the index.
$ git reset --soft HEAD^

# Reset the latest commit, and leave the changes in the working directory
$ git reset HEAD^

# Undo add, move the changes from the index to the working directory
$ git reset

# Reset the latest successful pull or merge
$ git reset --hard ORIG_HEAD

# Reset the latest failed pull or merge
$ git reset --hard

# Reset the latest pull or merge, into a dirty working tree
$ git reset --merge ORIG_HEAD

You can do more things with reset, but the above covers the typical cases. And now to the last thing, git revert. What does it do? git revert creates a new commit that is the opposite of the commit it names.

# Show the commits
$ git log --oneline
4717a5c new line
7e38e95 added tapir file

# Revert the commit named, 4717a5c, and commit it.
$ git revert 4717a5c

# Revert the HEAD commit, but don't commit it
$ git revert -n HEAD

Git is incredibly flexible and lets you control everything if you want to.

Inside Git

This is an exploration into what is going on when I run some basic git commands. We start out by creating a new repository. git/object is the directory where git stores all its objects, and it is empty initially.

$ mkdir myrepo
$ cd myrepo/
$ git init
Initialized empty Git repository in /Users/andersjanmyr/tmp/myrepo/.git/
$ find .git/objects -type f     # find all files in .git/objects
$ 

When a file is added to git it gets stored in the .git/objects directory under the name of its hash. The first two characters of the hash is used as the name of a subdirectory and the rest become the file name. Worth noting is that the hash uniquely identifies its content, so were you to run the commands on your computer, your results should be identical.

$ echo "A tapir has 14 toes" > tapir.txt
$ git add tapir.txt
$ find .git/objects -type f
.git/objects/12/a93608760777f50380a94b52e1b54ec69f4743
$ git hash-object tapir.txt
12a93608760777f50380a94b52e1b54ec69f4743

If you try to list the contents of the file, you are out of luck since it is stored in a binary format, you should instead use the git command git cat-file. The file above is a blob and its contents is what can be expected.

$ cat .git/objects/12/a93608760777f50380a94b52e1b54ec69f4743
xK??OR02`pT(I,?,R?H,V04Q(?O-?zi$ 
$
$ git cat-file -t 12a93608760777f50380a94b52e1b54ec69f4743
blob
$ git cat-file blob 12a936   # Using the first part of the hash is enough
A tapir has 14 toes
 

Even though the file is in the .git/objects directory it is not committed yet and it cannot be read by the high-level git commands such as git log. git status on the other hand will show that the file is staged, or in the index.

$ git log
fatal: bad default revision 'HEAD'
$ git status
# On branch master
#
# Initial commit
#
# Changes to be committed:
#   (use "git rm --cached <file>..." to unstage)
#
# new file:   tapir.txt
#

When I commit the file, two more objects are added to the .git/objects directory

$ git commit -m "added tapir file"
[master (root-commit) 7e38e95] added tapir file
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 tapir.txt
$ find .git/objects/ -type f
.git/objects//12/a93608760777f50380a94b52e1b54ec69f4743
.git/objects//7e/38e95d328287ea9d234a2affc4ed9e4510435a
.git/objects//e8/493a7e63154350f8c3d08a42e759132d9d2a39

One is tree object and the other is a commit.

$ git cat-file -t 7e38
commit
$ git cat-file -t e849
tree
$

The commit contains the information that was recorded when I committed. Apart from the commit message and my personal info it contains a reference to the tree object that was created simultaneously with the commit.

$ git cat-file commit  7e38
tree e8493a7e63154350f8c3d08a42e759132d9d2a39
author Anders Janmyr <anders.janmyr@jayway.se> 1253590540 +0200
committer Anders Janmyr <anders.janmyr@jayway.se> 1253590540 +0200

added tapir file
$ 

The tree object is stored in binary format and cannot be completely read without the help of git ls-tree. Now I can see that it contains a reference to the blob that was created initially, the tapir.txt file.

$ git cat-file tree e8493a7e63154350f8c3d08a42e759132d9d2a39
100644 tapir.txt?vw???KR?NƟGC$ 
$ git ls-tree e8493a7e63154350f8c3d08a42e759132d9d2a39
100644 blob 12a93608760777f50380a94b52e1b54ec69f4743 tapir.txt
$

So how does git know what is the latests commit? In git lingo the latest commit is know as the HEAD. If I look inside .git/HEAD I see a reference and this reference points to the latest commit.

$  cat ./.git/HEAD
ref: refs/heads/master
$ cat ./.git/refs/heads/master
7e38e95d328287ea9d234a2affc4ed9e4510435a

The .git/refs directory is where all the references of git live, heads and tags.

$ find .git/refs
.git/refs
.git/refs/heads
.git/refs/heads/master
.git/refs/tags
$ git branch olle
$ find .git/refs
.git/refs
.git/refs/heads
.git/refs/heads/master
.git/refs/heads/olle
.git/refs/tags

Creating a new branch with git branch shows that the branch is added to the heads directory, switching to it will change the .git/HEAD contents.

$  cat ./.git/HEAD
ref: refs/heads/master
$ git co olle
Switched to branch 'olle'
$  cat ./.git/HEAD
ref: refs/heads/olle

Git, simple, but beautiful!

Fat is Better

I have recently had discussions with some colleagues about what architecture they prefer and, while they seem to favor thinly sliced services, I have come to the conclusion that the overhead that comes with slicing services thin is not worth the extra time that it takes to setup, verify and test the complex, internal communication that comes with this kind of architecture. Fat is better!

If I am designing a system that should work in a coherent way, I want it all in my big, fat, juice object model. This enables me to put the functionality where it is most cohesive and, therefore, gives me the best design possible. Every object should carry its own weight.

If there are external services they must, by necessity, be outside the model, but the internal representation of the external service should be inside my model.

An example of an architecture that relies on thinly sliced services is REST. REST is very elegant and it definitely has a place when publishing resources. But REST models are anemic. They rely on you to GET the information from the resource, do things to it and then replace the information of the resource with a PUT. It is CRUD for the web. It is not intended to take advantage of what is good in object-oriented and functional programming, like sending behaviors into an object and have it perform the calculations for you.

The elegance of map and reduce (fold) is the essence of functional programming. How do you model map and reduce with REST? You can't! Polymorphism and encapsulation is the essence of object-oriented programming. Where does it go when everything is a resource? It disappears!

I have worked on projects where the goal has been to design every little part of the system as a free standing module with its own life and versioning, that can be switched in and out, but the artifacts have mostly been deployed together and have rarely given any extra value standing on their own. But they have given us a lot of grief when we tried to build a DRY system.

So, a fat model is the way to go. How fat? As fat as possible, but no fatter. How fat is that? As always this is a judgment call but, err on the side of fatter.

If, by luck or skill, my fat system reaches a workload where it will have to be split over multiple processors or machines, it will not be very difficult to split the system, since the system will be well factored, cohesive and DRY!

Note: "Fat is better" is somewhat related to worse is better by Dick Gabriel

What Eric Evans Would Have Changed in the DDD Book.

Here are my notes of Eric Evans talk What I've learned about DDD since the book

Essentials

Emphasize that the collaboration with the domain experts is essential. It is up to you to show the domain experts how valuable their collaboration is. If they cannot see the value of participating in the project, they will not to a good job, and they will try to avoid it.

Always produce at least three bad models. These models help to emphasize what is important.

The chapters "Distillation of the Core Domain" and "Context Mapping and Boundaries" should have been moved to the start of the big since these are the most essential areas.

Changes to Building Blocks

Domain Events

There is one new Building Block and it is the Event Object. The Event Object is an object the represents an Event that is significant to a domain expert. A benefit of domain events is that they give clearer, more expressive, models.

The events can be used for: - Representing the state (history) of entities. - Decouple systems with event streams (publish-subscribe) - Enable high-performance system.

Aggregates

Evans want to emphasize som things of aggregates. The boundaries of an aggregate need to contain transactions, distribution and concurrency.

When modelling the aggregates it is important to not over-specify on what part of the aggregate the properties and invariants are placed. Even though they are commonly placed on the aggregate root, this is not essential and it gives you a freedom if you not specify it too hard and too early.

Strategic Design

Large-Scale Structure does not come up very often and would probably have been left out of the book.

It is important to not spread modelling to thin, and to focus on the core domain and to create a clean bounded context.

Collaboration Patterns

There is two new collaboration pattern called Partners. This pattern differs from the other collaboration patterns in that it is cooperative and mutually dependent.

Another pattern is the Big Ball of Mud"_. This pattern is known as an anti-pattern since it implies that the code is just fixed as things go. This pattern is utterly pragmatic but the point Evans is making is that if there is a Big Ball of Mud in the system, it is important to define the boundaries of it, so that it doesn't spread to other systems that are more rigorously architected.

Context Mapping

• What models do we know of?
• Where does each apply? Define boundaries in words?
• Where is the information exchanged?
• The service interface may define a context boundary.

DDD and SOA

• The service interface must be defined in some context.
• Internals also, but often not the same one.
• The service interface may define a context boundary.

Precisions Designs are Fragile

Sophisticated design techniques are wasted in a ball of mud. It must be isolated with an Anti-Corruption Layer.

Not all of a large system will be well designed. Figure out what part of the system is most critical and benefits most from a really nice design.

Notes on The User Illusion, part Two

This is part two of a summary of the fantastic book. The User Illusion

Our conscious mind consists of symbols that map into the rest of our mind. Every second more than 11 million bits are reduced to less than 50 bits of meaningful information at apparently no time at all.

Half a second before we try to do something consciously our brains has started its activity. Half a second!

Experiment

• Sit down and hold one finger up.
• Whenever you feel like it, bend your finger.

Half a second before your finger is bent, your brain has started its activity! It does not feel like half a second does it? There is no way! 0.1 second is more likely. But the data has been reproduced and it is undeniable.

The consciousness of wanting to do something appears almost half a second after the brain has initiated its activity.

Our actions are initiated unconsciously.

Benjamin Libet

Benjamin Libet performed some experiments, in the sixties, while a friend of his was performing brain surgery. Libet stimulated the brain with electrical impulses and found that if the brain was stimulated less the half a second, it was not noticed at all. If the brain was stimulated more than halv a second the patient felt it. It appeared to the patient as if a certain area of his body had been touched, since there are no normal sensors inside the brain.

Libet then performed some other experiments. He stimulated a part of the brain that related to the left hand and at the same time he stimulated the right hand and the patient was to say which hand was stimulated first.

The hand had to be stimulated half a second after the brain to give the appearance to the patient that they where stimulated simultaneously.

The conscious experience is projected back in time so that the conscious mind believes that they are almost simultaneous. The brain is fooling itself.

The conscious is delayed, but it does its best to hide it. For itself. It is very convenient since it gives the mind the time to perform the reduction of the sensory data to what is needed to get a full experience.

Everyone knows that it doesn't take half a second to remove a hand from a hot stove, but it takes half a second to get the conscious back-dated experience.

So does this mean that we don't have free will? Libet does not think so. He says that there is time for the conscious mind to veto the unconscious decision.

The conscious is not a top-level unit that gives orders to its underlying processes. It is a selecting mechanism that chooses between the different options that the unconscious provides.

As long as the unconscious action-proposals work in sync with the conscious thoughts and feelings, they are hardly noticed. They are merged together with the conscious motives that take all the credit! --Harald Høffding

The conscious veto is often associated with an uncomfortable feeling, we usually feel best when we are not making conscious decisions. Compare this with for example Flow

I and Me

Nørretranders defines I as the conscious mind and Me as the rest of us.

I have free will, but it is not my I that has it, it is my Me.

The conscious mind is allowed to use its veto in situations where the Me allows it to. In situations where speed is critical, for example when something is threatening us, the conscious mind is left out of the loop.

A comfortable state of mind is when the Me is allowed to do what it does automatically without interference of the I. This state is neither associated with nervousness nor shyness, but with comfort and carelessness.

The User Illusion

The user illusion is a term coined by Alan Kay of Smalltalk fame. He used it to describe the metaphor of the desktop that was invented by him and his colleagues at Xerox PARC. His great idea was that it does not matter what really happens inside the computer as long as the interface presented to the user is useful and consistant.

Nørretranders argues that our consciousness is our user illusion. Our consciousness is our map into ourselves and our possibilities to affect this world. I am the user illusion of Me.

The conscious mind may appear when the brains' simulation of the world has become complete enough to require a model of itself. --Richard Dawkins

Experiments with split-brain patient has shown that the mind doesn't think twice about making up a story that is consistent with something that does not have anything to do with reality. An example of this is:

• A split brain patient is shown two images
  • The right eye was shown a snowy landscape
  • The left eye was shown a chicken's foot.
• The patient pointed to two images.
  • The right hand a snow-shovel.
  • The left hand a chicken.
• When the patient was asked why he pointed at the two images:
  • He said that he picked the chicken because it matched the chicken's foot
  • And that he picked the shovel because you need the shovel to clean out the chicken house.

The mind lied without hesitation to make the selection of images consistent.

Our consciousness is delayed to allow it to perform the required calculations. First we sense, then we simulate, then we experience. But we have no notion of the simulation. Consciousness is depth appearing as shallow.

Learning and Knowing

There are lots of things that we know how to do that we cannot explain. How do you ride a bike? How do you walk? How does your mother look? Its impossible to explain, but simple to know, once it has been learned.

The I's role in learning is to have the disciplin to learn by repetition, and the foresight that when I have learned this I will enjoy practicing it. The consciousness, usually, gets in the way while learning and, certainly, gets in the way when we practice what we have learned.

Science, usually, clarifies what we already know. It explains what we already know but cannot explain to each other.

Science is just refinement of everyday knowledge --Albert Einstein

The relationship between the conscious learning and the non-conscious ability is similar in both practical areas, such as ballet, and theoretical areas such as science; in both cases we must put in a lot of hard work for something we don't completely understand, but that we still can share with others.

The Rise of Consciousness and Religion

Nørretranders brings up another researcher, Julian Jaynes, who argues that the consciousness of man didn't exist 3000 years ago. He has various theories to prove this. Among other things he brings up the Oracles of Ancient Greece and the voices religious people hear as examples of people who are acting without a consciousness. A pre-conscious man is just a me, and a conscious man thinks he is just an I.

He also makes an interesting point about religion. The I has to recognize that there is something greater than itself. But it is not a god it is the me. God is the part of man that the I cannot explain. Religion is too important to be left to the religious. It is the struggle between the carefree me and the worrying I.

The Limits of Consciousness

Most of the processes that go on inside our bodies we know nothing about. We cannot feel the blood flowing in our legs or our immune system attacking viruses, until the problems become large enough for the body to raise the temperature to increase the effect. We cannot hold our breaths long enough to kill ourselves. The unconscious part of us wont let us

The Beginning of the Universe

In the Universe there is as much negative matter as positive matter. The sum of it all is Zero. According to the laws of quantum mechanics it is possible for Nothing to split into something for a very short while. The less it is the longer it is allowed to exist. So, it the sum of everything in the Universe is Nothing can exist forever.

Emergence

Emergence is the appearance of group characteristics when the number of elements increase. An example of this is Temperature. There is no temperature when there are only a few molecules but when the number of molecules increases the Temperature property emerges. In the book Gödel, Escher, Bach, Douglas Hofstadter talks about the emerging properties of the brain, the thoughts that cannot be described at a lower level.

Gödels proof suggests the possibility that an view from a higher level may have an explanation capacity that is totally missing at a lower level. --Douglas Hofstadter

A larger system consisting of simple rules can show characteristics that cannot be deduced from the rules.

It is impossible to know how yourself or another human will react, because it requires you to have access to all the information yourself or this other person has had and this is impossible since human being mostly function unconsciously.

The Liars Paradox, "I'm lying" is not a liars paradox. It is the truth about our consciousness. --Thor Nørretranders

Notes on The User Illusion, Part One

The User Illusion, is a magnificent book. It is written by the danish author Thor Nørretranders. The Swedish title of the book, "Märk Världen", is a better title. "Märk Världen" means Notice the World it can also be a pun, "Märkvärdigt", means Astonishing.

The popular usage of the term information has been perverted from the original meaning of the word. The original meaning of the word is similar to the word data. It is something that can be transmitted, stored or transformed. It does not have anything to do with meaning. In popular usage the word information is often interpreted as something that has an actual meaning and not just raw data that may or may not mean anything to anyone.

Thor Nørretranders refers to information as data. It does not matter if a conversation over the phone is gossip or if it is an explanation of the theory of relativity. It is information nonetheless. He claims that random information commonly referred to as chaotic as containing more meaning than ordered information. To support this claim he gives a simple example.

A random sequence of coin-tosses (101010101111) cannot be reduced to anything but the same sequence of coin tosses, while a regular sequence (111111111111) can be reduced to the simple statment: 12 ones. Therefore the 12 ones contain less information than the coin-tosses.

Another interesting aspect of this is that it is impossible to know in advance if a sequence can be reduced or not until the actual reduction has been done. This is related both to Turing's halting-problem and to Gödel's incompleteness theorem.

Logical Depth

A conclusion's logical depth is a measurement of its meaning, its value. The harder it is for the sender to reach a conclusion, the larger is its logical depth. The more "calculation time" he has used the greater is its value, since the receiver of the conclusion is relieved from having to perform the same calculation.

Logical depth can informally be defined as the number of steps in a conclusion or causal chain that connects something with its probable origin. Logical depth describes complexity.

A table of moon phases can be calculated by a simple formula, but it takes time. A living organism can be specified by a few genes but it takes a long time to develop the complete organism.

Chaos and disorder cannot be reduced to something less. The shortest program is identical to it all. They have no logical depth.

It is difficult to make things look easy. Clarity demands depth.

Communication and Exformation

To communicate is to take information (a mental state) in the sender, reduce this state to a message that can be transferred over a channel to the receiver, the receiver then interprets this message as information (his own mental state).

The explicitly removed information, Nørretranders calls exformation. A statement has depth if it contains a lot of exformation. That is, information that existed in the sender, but was purged while composing the message, and is no longer present in the final result.

There is no possible way to calculate the exformation of a message from the contents of the message. This can only be found in the context of the message. The sender forms the message so that it refers to a common context allowing the receiver to re-create the meaning from this context. A good communicator not only thinks of himself, but also of what the receiver is thinking about.

Meaning is purged information, removed information, unneeded information, exformation.

Information is not needed to transfer exformation. An example is an agreement between a child and his parents that instead of calling every week, he will only call if he is in trouble. Every week exformation is communicated, all is well, by not sending any information at all.

To communicate well we must be able to take the information in our heads and create meaningful symbols of it. These symbols should represent a common understanding so that the recipient of the symbols can induce the same understanding that we have.

Meaning and Consciousness

What we consciously experience in any given moment, limits itself to a very miniscule part of the flood of information that flow through our senses. --Manfred Zimmermann

The bandwidth of our conscious is less than 50 bits/s. Our senses take in around 11 million bits/s with the vision at 10 million bits/s, touch - 1 million bits/s, hearing and smell - 100 000 bits/s, and taste - 100 bits/s.

Most of what we experience, we can never tell someone else. We experience millions of bits but can only verbally communicate decades.

But we are not limited to verbal communication. Just as our senses can recieve 10 millions of bits our bodies can also communicate. Our bodies send out 100 000 bits/s through body movements, voice modulations, facial expressions, etc. It is just not possible to recieve all this information consciously. This is why there is no substitute for face to face.

A good bedtime story is good since it induces feeling in the adult that the child is able to pick up and react to. The excitement, sadness, and happiness induced by great tales such as The Ugly Duckling, by H.C. Andersen is what makes them great. The tale creates a much wider communication channel than a tale that means nothing to the adult, since the emotions emitted from the adult induces the same feeling in the child at a much higher bandwidth than simple words can communicate.

The unconscious is not hidden from anyone, but the person, who for himself tries to hide the sides of himself that were disliked by the people who he loved while growing up.

Others know more about us than we do.

Most of the information that pass trough us is never experienced consciously, although this information has a noticable effect on our behavior.

The Unconscious

When we meet a new person, we usually make a really quick decision whether we like the person or not. We say, "You nerver get a second chance to make a first impression!" and "We don't have chemistry". These decisions are made so quickly that they cannot possible be made by the limited understanding of our conscious. This is indirect evidence of high-speed channels that operate unconsciously.

The conscious is a much smaller part of our life than we are conscious of, because we cannot be conscious of that which we are not conscious of.

Our conscious is like a flashlight and wherever it looks there is light. The obvious conclusion of the flashlight is that it is light everywhere. We are conscious less than we think, because we cannot be conscious of not being conscious.

I am sure that there are no words in my conscious when I am really thinking. --Jacques Hadamard

Real thinking is not a process that is performed consciously.

Thoughts die the moment that they are put into words. --Schopenhauer

When was the last time you ate fish? Yesterday? Last year? Never?

The moment you started thinking about this, your conscious mind let go. How did you come up with the answer you gave? Did you think through all the meals you have eaten in your life or did it just pop into your head?

Summary part one

The first part of the book sets the stage for the rest of the book. It introduces, and defines, the words: information, exformation, and logical depth. It also introduces the notion of meaning, consciousness and unconsciousness.

Our conscious can contain less than 50 bits per second. While our senses take in more than 11 million. Our brains process the 11 million bits of information and reduces them down to 50 meaningful bits, symbols that map into the rest of our brain and knowledge. But processing takes time even for something as able as our brains. But isn't our conscious perception immediate? No, it's not, but it appears to be! The second part of the book and the second part of this summary deals with this discrepancy.

Notes on The Secrets of Consulting

I finally found time to read The Secrets of Consulting by Gerald Weinberg. With all books with titles like, How to Win Friends and Influence People, I very reluctantly start to read them, because, they sound like books on learning how to be manipulative and I don't want to be like that. But when I finally start reading them, after many independent recommendations from people I trust, I find them to be very down-to-earth and not at all manipulative. They are more about how to become a sympathetic human being and because you are, to become successful in other areas too.

The book is full of good ideas and I am going to mention a few that I like from every chapter in the book.

Why consulting is so tough?

The quotes I like best in this chapter are

You'll never accomplish anything if you care who gets the credit.

This is very true. I've seen this many times. People who care about credit, usually, wants a little more than they deserve and this will piss of other people who also wants the credit. Meanwhile, the people who don't care about the credit, just keep on working, making things happen.

When an effective consultant is present, the client solves problems.

And the funny part is that it does not matter as long as I feel that I am doing a good job.

Cultivating a Paradoxical Frame of Mind

The answer to Get it done in the shortest time possible is

What are you willing to sacrifice?

This implies that everything is a tradeoff. The most common tradeoffs are Now versus Later and Risk versus Certainty. Another good reply when someone wants to get something done is

We can do I it - and this is how much it will cost.

This is a very good reply since it shows that you know what you are talking about and that you are not just saying something to get the contract.

Being effective when you don't know what you are doing.

The most important thing to remember is that:

It is always a people problem.

People always want to catch a consultant with a lie, so:

We ought to bend over backwards to understate our qualifictions, but    insecurity makes us all victims to occasional exaggeragtion.

Seeing what's there

If you use the same recipe, you get the same bread.

Learn from history. If you don't listen to the mistakes the client has made you are likely to make the same mistakes.

Study for understanding, not for criticism.

A client is not very likely to help you, if you criticise everything he has done. Look for what you like in the present situation and comment on that. Someone will always comment on what's bad.

We may run out of energy, or air, or water, or food, but we'll never run out of reasons.

People never run out of reasons. We can think of a million reasons to rationalize our behavior and none of them may be the true reason. We may not even know it ourselves.

The name of a thing is not the thing.

This is so very true. Naming is really, really important! But we should always be aware that a name is just a label and there is always more to it than that. If we think the name is the thing, we may miss important points.

When you point a finger at someone, notice where the other three fingers are pointing.

It is always easier to blame someone else.

Seeing what's not there

It is as important to see what's missing as it is to see what is there. Use laundry lists to remember what to look for.

If you can't think of three things that might go wrong with your plans, then there is something wrong with your thinking.

It is easy to become to narrow-minded when it comes to your own thinking.

Words are often useful, but it always pays to listen to the music, especially your own.

Be aware of congruent words and emotions.

Avoiding traps

What you don't know may not hurt you, but what you don't remember always does.

Set triggers to help you remember. Triggers may be anything from songs to one-liners, jokes and mental pictures.

It ain't what we know that gets us into trouble, it's what we know that ain't so.

Never be too sure of anything.

Amplifying your impact

Characteristics of a good consultant:

• Your task is to influence people, but only at their request.
• Your task is to make people less dependent on you, rather than more.
• Try to Jiggle. The less you actually intervene, the better.
• If your client wants help solving problems, you are able to say no.
• If you say yes but fail, you can live with that. If you succeed, the least satisfying approach is when you solve the problem for them.
• More satisfying is to help them solve the problem in such a way that they will be more likely to solve the next problem without help.
• Most satisfying is to help them learn how to prevent the problems in the first place.
• You can be satisfied with your accomplishments, even if the client don't give you credit.
• Your ideal form of influence is first to help people see their world more clearly, and then to let them decide what to do next.
• Your methods of working are always open for display and discussion with your clients.
• Your primary tool is merely being the person you are, so you most powerful method of helping other people is to help yourself.

Gaining control of change

Feedback

Fords Fundamental Feedback Formula
1. People can take any amount of water from any stream to use for any purpose desired.
2. People must return an equal amount of water _upstream_ from the point from which they took it.

Believing in what you do

Would you place your own life in the hands of this system?

How to make changes safely

Nothing new ever works.

I have come to this conclusion myself and I heard Joel Spolsky mention a harsher version of this on the Stackoverflow Podcast. He said Nothing ever works! not just new things.

Trust everyone, but cut the cards.

Fundamental skepticism is necessary.

If you must have something new, take one, not two.

When trying out something new, try one thing at the time otherwise you will not now what fails when it fails.

It may look like a crisis, but it the end of an illusion.

When a crisis occurs it is a sure sign that something has not been right in the first place.

What to do when they resist

The first thing to do with resistance is to appreciate it. If you get no resistance something is clearly wrong. When facing resistance Get it out into the open. Name the resistance in a neutral way. * Locate the nature of the resistance.

You can make a buffalo go anywhere, just so long that they want to go there.

Clients tend to overestimate unspoken negative factors and to forget positive ones.

Good questions for defusing potential resistance are:

• Is there anything you would like to change about this plan?
• What do you like best about this plan?

• What is the one thing that you want to be sure does not change?

  People who are realistic about risks don't become consultants.

• Step away, if you feel the fight cannot be won.

Marketing your services

1. A consultant can exits it two states, Idle or Busy.
2. The best way to get clients it to have clients. It is best to look for new business when you have business.
3. Spend at least one day a week getting exposure.
4. Clients are more important to you than you can ever be to them.
5. Never let a single client have more than one-fourth of your business.
6. The best marketing tool is a satisfied client.
7. Give away your best ideas.
8. It tastes better when you add your own egg.
9. Spend at least one-fourth of your time doing nothing.
10. Market for quality, not quantity.

Putting a price on your head.

1. Pricing has many functions, only one of which is the exchange of money.
2. The more they pay you, the more they love you. The less they pay you, the less they respect you.
3. The money is usually the smallest part of the price.
4. Pricing is not a zero-sum game. My gains doesn't have to be their losses.
5. If you need the money, don't take the job.
6. If they don't like your work, don't take their money.
7. Money is more than price. If the clients have paid in advance they are more likely to be prepared for the job.
8. Price is not a thing; it's a negotiated relationship.
9. Set the price so you won't regret it either way.
10. All prices are ultimately based on on feelings, both yours and theirs.

How to be trusted

1. Nobody but you cares about the reason you let another person down.
2. Trust takes years to win, moments to lose.
3. People don't tell you when they stop trusting you.
4. The trick of earning trust is to avoid all tricks.
5. People are never liars - in their own eyes.
6. Always trust your client - and cut the cards.
7. Never be dishonest, even if the client requests it.
8. Never promise anything.
9. Always keep your promise.
10. Get it in writing, but depend on trust.

Lessons from the farm

1. Never use cheap seeds.
2. A prepared soil is the secret of gardening.
3. Timing is critical.
4. The plants that hold the firmest are the ones that develop their own roots.
5. Excessive watering produces weakness, not strength.
6. In spite of your best efforts, some plants will die.

All in all, this is book full of good advice that is worth reading for anyone, not only consultants. Worth noting is that the consultants in this book are very far from the resource-consultants that are the common case in Sweden. The consultants this book refers to closer resemble what we know as management consultants, although it is not an exact match.

Markdown

Markdown is a simple syntax for writing text documents. It is better than HTML since it does not contain as much clutter. HTML may be used in a Markdown document but block level elements must be surrounded with blank lines.

Headers

Headers come in two types.

• Underlined headers, =====, and -----
• Prefix headers, which uses # as prefix, # = h1, and ### = h3

Emphasis

• Emphasis can be shown with * or with _ .
• If you double them up they will become strong instead ** and __

List

• Unordered lists are indicated with *, + or -
• Ordered lists are indicated with number followed by period, 1.

Lists may be span several lines and may also use hanging indents. To wrap the list items in paragraphs simply leave a blank line between item lines.

Blockquotes

Blockquotes are indicated by prefixing lines with >. They may be nested > > . If you only prefix the first line in a paragraph, everything up to the next blank line will be part of the blockquote.

Code

Markdown produces literal markup if the lines are indented with one tab or four spaces. This kind of code will be wrapped with <pre><code>

Lines

Lines may be indicated by putting three or more asterisks (***) or hyphens (---) on a line by itself.

Links

Markdown support both inline and reference links.

Inline links

• Absolute [text for link](http://url.to.site)
• Relative [text for link](relative_url)
• With title [text for link](relative_url "my title")

Reference links

• [text for link][anders]
• Same text and id [anders][]

The referenced link ids must be defined somewhere else in the document like this:

[anders]: http:/anders.janmyr.com "Optional title"

The link definitions are inserted into the actual links and removed after the page is processed.

Automatic links

URLs surrounded with angle brackets <> are automatically inserted as links this may be used both for links and email:

• <anders.janmyr@email.com>
• <http://anders.janmyr.com>

Images

Images are inserted with link syntax, preceded with an exclamation point !.

• ![Alt text](/path/to/img.jpg)
• ![Alt text](/path/to/img.jpg "Optional title")
• ![Alt text][id]

Escapes

Most of the Markdown special characters such as * and - may be escaped by preceding them with a backslash \.

Ruby Method Lookup

There are three different kinds of methods containers in Ruby, Classes, Modules and Singleton Classes. When a method is invoked all of them are searched for the first matching method to execute.

A class may contain methods

class Mammal 
  def breathe
  end
end

A class may inherit its methods from one other class.

class Tapir < Mammal
end

A class may also mix in methods from several modules.

class Tapir 
  include Swimmable, Runnable
end

Methods may also be defined on the object itself, this is actually the anonymous Singleton Class that is available to every object.

kjell = Tapir.new
# kjell can not only swim, he can crawl
def kjell.crawl 
end

So what happens if all these different method containers define the same method? How will the interpreter know what to execute?

When a method is called:

• The Singleton Class is searched. —the anonymous class of kjell
• The modules included in the Singleton Class is searched. —in this case none.
• The class is searched. —this is the Tapir class
• The modules included in the class is searched in reverse order of inclusion, the modules override each other as they are included. —Runnable, then Searchable
• Then the search continuous in the superclass of the class and the modules included in it.
• If the search reaches the top of the hierarchy and no method is found, method_missing is invoked on the initial object and the search for method_missing follows the same path.

That’s almost it. There is one caveat. If the Singleton Class is a meta-class, that is the singleton class of a class, then the class belongs to a meta-class hierarchy that parallels the hierarchy of the normal classes. This allows for inheritance of class methods and gives us the ability to override class methods such as new. The lookup path is changed to search the meta-class’ superclass after step two, before the class of the class (i.e. instance methods of class Class).

Notes on Strangers to Ourselves

Strangers to Ourselves by Timothy Wilson is a book in the same spirit as the User Illusion by Thor Nørretranders. Both books are about how our unconscious mind plays a much bigger part in our lives than our conscious mind gives it credit for.

Our senses register 11 million bits per second and our conscious mind can only deal with 40. What happens to the other 10 999 960 bits is the topic of this fantastic book!

Adaptive Unconscious

In one study a woman suffering from lack of short-term memory was visited several times by a researcher. Every time he visited her, he had to re-introduce himself, since she didn’t recognize him. Then one time he pricked her with a pin when they shook hands. The next time he visited her, she didn’t recognize him, but she refused to shake his hand.

Attention and selection: the non-conscious filter. We become aware of new things even though we are in the middle of doing something completely different. For example: I am talking to someone at a dinner somewhere and then I hear someone mentioning my name in a different group talking amongst themselves. Suddenly my attention shifts to that of the other group

Interpretation: the non-conscious translator. Our preconceptions affect how we interpret situations. For example: If you after hearing something positive about someone it is a lot more likely that you will interpret their actions as good than if you haven’t heard anything positive. Studies with teachers show that teachers that were told that a random selection of students were likely to succeed, were affected so much that the selected students actually improved more than the other students.

Feeling and emotion: the adaptive unconscious as evaluator. In a study where the subjects were given cards from 4 decks of cards, A, B, C, and D, they were able to select the best decks on “gut-feeling” even though they could not verbalize why they selected as they did.

Unconscious goal setting. When we are playing games with our kids we may select not to win the game to make the kids feel happy and ourselves fell gracious. After doing this a lot our unconscious may automatically select for us not bothering the conscious with the burden to select.

Who’s in Charge?

Is our conscious in charge or are the non-conscious processes in charge? Some believe that our conscious is like a president, and makes all the executive decisions about what our non-conscious processes should be doing, others believe that our consciousness is more like a child playing demo games. They think that they are playing, but in reality it is just a simulation that they can do nothing about.

If I feel hungry and decide to go to the kitchen to eat a sandwich, it may feel like I make a conscious decision but, it may well be that my desire to eat arose non-consciously and triggered both my conscious thought and my trip to the kitchen. The consciousness may just be an illusion.

The truth is probably somewhere in between. Our consciousness, differs from our adaptive unconsciousness in several important ways.

Adaptive unconsciousness	Consciousness
Multiple systems	One system
On-line pattern detector	After the fact check and balancer
Concerned with now	Long view
Automatic (fast, unintentional, uncontrollable, effortless)	Controlled (slow, intentional, controllable, effortful)
Rigid	Flexible
Developed and mature	Slower to develop
Sensitive to negative information	Sensitive to positive information

Knowing Who We Are

Personality has been defined as the psychological processes that determine a person’s “characteristic behavior and thought”.

We probably have two personalities, one conscious and one unconscious. The conscious is measurable through questionnaires, while the unconscious is only measurable through indirect techniques.

As an effect of our two personalities we are also likely to have dual motives and goals. If our motives and goals and thus our two personalities correspond well, we are more likely to feel emotionally well.

Knowing Why

Our conscious is fantastic at making up causes for why we act the way we act. In a study with a split-brain patient (a patient who have had their brains split in half), researchers showed the patient pictures, one for the left eye and one for the right. At one time he was shown a picture of a snow scene to his left eye and a chicken claw to the right eye. He was then asked to pick up cards that related to the pictures. He picked a shovel with his left hand and a chicken with his right. When he was asked why he picked the cards, his answer was: “I saw a claw and picked a chicken, and you have to clean out the chicken shed with a shovel”. Our speech center is in our left hemisphere so the patients brain knew that it had seen a claw. It had no idea, however, why he had picked a shovel so it made up a reason.

Why do we do the things we do? We really have no idea. When an idea pops up into our head we may trace the thoughts causally back until we reach something that seems to be a reasonable cause of our thought. But, this thought may just be made up by our conscious mind to get a felling of control.

If I decide I want a sallad for lunch today and I am asked why, I will probably justify it with, something like, I am trying to lead a healthier life, etc., etc. But the real reason may be that I saw an obese person earlier this morning and he generated the thought later that day.

The theory is that we have no access to our mental processes, only to the mental contents that are the results of those processes.

Studies made in this area, have shown that a stranger may be as good as we are at predicting why someone did something. It may be as accurate to call a random person up, give them some data about what you have done during the day, and then ask them how you feel as to try to figure it out yourself!

Even if we, personally, have access to more information, there is no reason that this extra information will make our conclusion any more accurate.

Knowing How We Feel

Our feelings have always been believed to reside in our conscious, what good would they do if we didn’t notice them? But, not even our feelings are sure to be completely conscious. The author gives the argument that, sometimes we feel something and not until later do we realize that we didn’t have that feeling at all. Wilsons example is a couple of people that talk about a horse they owned as a kid, one of them says, “Good, I hated that horse!”, and the other person realizes that she has always hated it too. Even though she didn’t acknowledge it at the time.

Another example is that when we experience something scary, like losing control over a car, the feeling of fear doesn’t strike us until after the incident is over. Wilson argues that we get an unconscious feeling and only afterwards does the feeling appear in our conscious.

Other feelings that we fail to recognize are prejudice feelings. We think that we are very liberal but, at a deeper level we feel the feelings anyway and others can tell even if we don’t allow ourselves to see it.

The reason that we have these unconscious feeling is that they react a lot faster than our conscious feelings do. When we see a stick lying on the ground we may first react scared, thinking it is a snake, only to realize afterwards that it was only a stick and move on. If it is a snake the faster reaction may say our life.

Knowing How We Will Feel

If it is difficult to know what we feel at the time, how are we to guess what we will feel in the future? When we try to predict how we will feel in the future we usually exaggerate the feeling. How would you feel if your wife died? I would never get over it. How would you feel if you won twenty million dollars. I would live happily ever after, doing whatever I please.

Neither of these replies are very likely. There are a number of reasons why we exaggerate our future feelings. First, they are thought of in isolation, we don’t take into account all the other things that will happen at the same time as the events take place that we are imagining. Second, as soon as something happen to us, we start to internalize it. If I win the lottery I will be happy for a while but, after a while this will be the status quo that I compare things against.

This implies that we know very little about ourselves. Is there hope for us to improve this?

Introspection and Self-Narratives

It is often thought that to introspect our feelings, to think about why we feel a certain way, is a good approach to self-improvement. But this is not always the case.

The problems with introspection is that we may try to make too much out of what we find. If we are asked why we love our spouses and try to make a list of reasons why, we run the risk of believing that this is the whole story and not just a small subset of it. It may not even have anything to do with why we feel the way we feel. When we try to articulate our feeling we diminish them.

He who deliberates lengthily will not always choose the best. —Goethe

So, are we supposed to ignore why we feel the way we do and just act on impulses? No, it is important to distinguish between informed and uninformed gut-feelings. The trick is to gather enough information to develop an informed gut-feeling and then not analyze that feeling to much. We should let our adaptive unconscious do the job of forming reliable feeling and then trust those feeling, even if we cannot explain them entirely.

If we try to imagine, in detail, the situation that would occur if we acted on our feeling, perhaps we would be better at recognizing our true future feelings

Looking Outward to Know Ourselves

To complement knowing ourselves by introspection, we can look outwards instead. We can look at the research done in psychology. For example, in one study, a group was tested for automatic prejudice. The study showed that people unconsciously made different decisions when they were not given enough time to think over their replies consciously. To know that our unconscious opinion may be different than our conscious one is good knowledge to have.

To try to figure out what we are like by observing how other people react to what we do is an approach that isn’t very fruitful. Our observation of the other person gets filtered by our unconscious and it is not at all unusual to think that someone admires you, while they think you are a complete idiot.

A better approach is to ask others what they think and use their description as a clue to finding out how we are. But the people you ask have to be very honest.

Observing and Changing our Behavior

Despite years of research on self-perception theory, there is an enduring question: Is the self-perception one of self-revelation or one of self-fabrication. Self-revelation is good in that it helps us understand ourselves, while self-fabrication isn’t since it infers things that didn’t exist before.

In the end, the only way to change how we are is to change how we act. Since acting in a way you want to act will make you a person who acts like you want to act.