Atlas Blog

This introductory text can be changed from the options panel.

Big Data for Big Cities: A Solution to Auckland’s Traffic Congestion

If people across the world were to agree on one thing, they would agree on that traffic congestion is bad for everybody. Traffic congestion is a major drawback of big cities. It negatively affects the environment through increasing CO2 emissions, slows down the economy through lowering productivity and mobility, stimulates social issues such as road rage, wastes a considerable amount of time, and the list goes on.

Increases in urbanisation and migration have exacerbated the problem of traffic congestion worldwide. This seems to be the case in Auckland, the largest city in New Zealand. Auckland population increased by 21% from 2001 to 2013. This significant growth in population resulted in more people needing to commute. Unlike other fast-growing cities, however, Auckland’s transport system does not seem to accommodate its growth.

Traffic congestion in Auckland has become an unbearable nightmare for many Aucklanders. The morning bus going from Huapai to the CBD leaves at 6.30am and arrives at 8.33am (NZ Harald). A couple of years ago, car drivers travelling in the morning from Albany to the CBD used to spend less than 30 minutes in the motor way whereas now it takes up to an hour to get to Harbour Bridge. Aucklanders are increasingly frustrated and dissatisfied with the performance of the city’s transport system (Wilson, NZ Harald).

In attempts to improve the transport system, Auckland Transport (AT) has been exploring long and short term solutions. These included upgrading existing facilities, “bus lanes, and different rail options including … a diesel shuttle, electric trains with and without double tracking” (Wade, NZ Harald). Unfortunately, these solutions were found to be costly, are faced with bureaucratic barriers, and/or require political motives (Wade, NZ Harald). Although Auckland Council and AT might have long-term plans to improve the system, a solution to reduce the current traffic congestion is very much needed.

There seems to be three approaches to reducing and controlling traffic congestion. The first approach is the use of physics to control the flow and movement of vehicles on the road. A classic example is the elimination of intersection that resulted in the idea of highways. Traffic engineers also influence the flow of traffic through using traffic lights, ramp stop lights/signs, merging lanes …etc. Most of these are already implemented in Auckland’s motorways. The cost of adding more of these traffic-management tools will probably be high and not exceed the benefit.

Another approach to reducing traffic congestion is introducing new policies. Road choices can be influenced by introducing an economic externality (e.g. taxing certain roads/lanes during peak hours) and/or incentives (e.g. tax exemption for jobs in rural areas and less populated towns). For example, the capital of Sweden, Stockholm, started charging about ₤2 on the use of bridges during rush hours, resulting in a 20% decrease in traffic congestion. In Auckland’s case, however, this approach does not seem suitable because of the bureaucratic and political barriers as pointed out by Ms Rose (Wade, NZ Harald).

The third approach is the use of high-tech solutions. These are devices built on IT systems and GIS technology to predict traffic flows and provide feedback on current traffic congestions. These devices might be good when there is plenty of alternative roads to use. The problem is that the device uses similar algorithms and same satellite, leading all device users to receive similar information and escape to the same direction which ends up creating traffic congestion.

Given the limited options AT has, an efficient solution would require looking into the current resources that are available and abundant. What is available and abundant is data… Big Data!

This article proposes a solution to the increasing traffic congestion in Auckland using marketing theory and Big Data analytics. The objectives include ensuring that the solution:

  • reduces the current traffic congestion in Auckland significantly;
  • is feasible; and
  • results in an efficient allocation of resources (minimise costs while increasing returns).


The reminder of this article is structured as follows. The next section introduces the solution and explains the theoretical background on which the solution is built. This is followed by an empirical illustration of the solution using traffic data collected in North Shore, Auckland. Then, the article points out the limitation, provides recommendations and concludes.


The Solution: Traffic Basket Analysis!

Theoretical background

Driving and consumption behaviours are similar in that choices and decisions are made to meet needs. Most of these choices and decisions are common and somewhat repetitive. One neglected cause of traffic congestion is the shared needs of road users. Humans live in structured and organised societies. Hence, geographic areas in big cities tend to be meaningfully clustered. There are commercial areas, residential areas, tourist areas, sport areas, and so on. Commuters move between these areas to achieve common goals (i.e. shopping, visit family, get to work…etc.). Thus, one would find a group of people making similar road choices. Examples include choosing to use the motorway to get to work, pick up children from school using the South-North side of the road, and use Road #10 to stop at the supermarket on the way back home. While these similarities between road users create traffic congestion, these also form patterns of co-decisions. The task is to identify associations between congested road choices and link the associated ones.

Market Basket Analysis is one powerful tool that marketers use to determine whether associations between choices/decisions are significant/strong, true, and co-occurring. In marketing, it is common to find associations between purchases of two, or more, complementary products. Examples include BBQ sauces in the meat section, laundry baskets with washing powders/liquids, bread and jams …etc.

Similarly, it is not unusual to find associations between two, or more, congested roads. A significant number of drivers using the Northern Motorway (SH1) also use Queen Street. Roads are complementary in nature; Figure 2 explains this. Travelling through Points B, C, and D complements the journey to Point E. But, what if we know that the association between Point A and Point E is strong and that, say, 80% of commuters departing from Point A go all the way to Point E?  Linking Points A and E directly makes the roads to Points B, C, and D substitute while it makes the road linking Points A and E complementary. The outcome should then be the removal of unnecessary traffic congestions at Points B, C, and D.

Empirical Illustration

AT has kindly made Auckland’s traffic data from July 2012 to February 2017 available on their website. The dataset contains traffic count data and statistics of over 7000 roads across Auckland. North Shore area is selected for this illustration. The focus here is on morning peak hours. There are three important steps required to perform the analysis and apply the solution to Auckland’s case.


First, the top 20 most-congested roads are identified and selected for analysis. This is to ensure that the effect is large enough to take actions; in other words, ensuring that the solution helps a large portion of road users. Figure 3 ranks the top 20-most congested roads in North Shore, Auckland by volume of traffic. During the morning peak hour, Beach Road is the most congested road in North Shore while Apollo Drive is #20 most congested road in that area.

Second, the association rules are applied to the 20 roads to ensure that association between roads are true so that the same associations will be found the next morning, next week, and next month …etc. One major limitation of this illustration is that the data are recorded at road-level instead of commuter-level. Thus, it is impossible to know where exactly the traffic goes from these congested roads. For the sake of this illustration, the following assumptions are made:

  1. not all road users leave the congested area (stay to work, live, or any other reason);
  2. traffic flows randomly to the nearest roads;
  3. most drivers travel from North to South (i.e. Albany to Davenport/CBD).


Random proportions of road users are assumed to travel through other roads. The closer the travel distance is, the more likely it is to be the commuter’s next destination.

Third, Zhang’s (2000) Measure of Association and Dissociation is used to examine the level of co-occurrence between associations. In other words, the measure indicates whether Road Y is used when Road X is used. There is a co-occurring association when Zhang’s Measure is positive and a dissociation when it is negative. The higher the outcome of Zhang’s Measure, the stronger the co-occurrence. Figure 4 presents the result of the analysis; the map shows only the co-occurring associations between congested roads, and illustrates how traffic congestion can be reduced.

Refer to Figure 4, each coloured line on the map represents an association/s with one congested road/s. For example, drivers who used Beach Road also used East Coast Road (Taka/Glenfield/E.C), East Coast Road (Silverdale/Redvale), and Apollo Drive. Some of these associations are more obvious than others. An example is the co-use (0.66) of Lake Road-Devonport and Hurstmere Road.


Now that we know the co-use of congested roads, our next task is to link these associated roads/areas. There are several ways to link two, or more, roads including cycling paths, bus routes, and special/smart lanes. For this to work properly, links between two congested areas should not be interrupted by intersections, bus stops …etc.. Linking the associated roads using the modern smart lanes/roads, such as the award-winning project SMART Tunnel in Malaysia, may be good for reducing traffic jams but are very expensive. An efficient solution is to offer a non-stop bus service. The non-stop bus goes back and forth using less congested roads to pick up and drop off commuters from two co-used congested roads/areas. For instance, the bus travels from Leak Road-Devonport to Sunnybrae Road and returns without stopping on the way.


One major limitation of the illustration in this article is that the data are at road-level. This makes it harder to know which roads were exactly used by commuters. In case AT has no access to such data (driver-level data), a survey may need to be conducted to obtain these data.


  • The non-stop bus should not be interrupted by stops (dropping off and picking up commuters);
  • The non-stop bus service should be frequent during peak hours (mornings & evenings);
  • The waiting time for the next non-stop bus should not be long;
  • The price should not exceed price tickets for the normal bus service;
  • The bus route should be selected carefully so that it takes commuters less time to get to their destination than using their cars;
  • The non-stop bus departs from a point where car parks are available (other than bus stations).



Auckland’s traffic congestion is a serious issue causing a lot of the frustration in the city. Although AT has a number of solutions to reduce the stress in the city, these solutions are either costly, have bureaucratic barriers, and/or lack political motives. Because traffic congestion is increasing by the day, a solution to control it and reduce it is very much needed. This article proposed the solution of using big data and marketing theory to reduce traffic congestion in Auckland. Applying the logic of the well-known market basket analysis to traffic congestion in North Shore, it has been found that congested roads are co-used.

The proposed solution meets three important objectives (stated in the beginning of this article). First, one can be assured that the impact of using this solution is significant. This is because the statistical analysis used in this solution includes the most congested roads (top 20) only. Hence, the outcome of the analysis will influence the largest portion of road users. Second, the solution is feasible as all that is required is 1) data analysis; and 2) offering non-stop bus service. Third, this solution results in a more efficient allocation of resources. The costs of traffic congestion are minimized through encouraging road users to use the non-stop bus service; Thus, less pollution, less frustration, and less time wasted on the roads. The returns are maximized through improving mobility on the roads, improving productivity (workers travel time is reduced), and charging for the non-stop bus service.

The article, therefore, concludes that linking co-used roads using non-stop bus service has the potential to remove significant amounts of unnecessary traffic jams, resulting in a more efficient transport system and, ultimately, large smiles on the faces of morning commuters.


We, at NeoRetailing, are happy to offer AT and/or Auckland Council our help in testing and applying this solution.


Share/like this article if you:

  • support testing and applying this solution; and/or
  • would like to let AT know about this idea/solution.


PS: Data presented in this article are for illustration purposes only.

What Did Last Year’s Big Issues in Retail Survey Predict Correctly?

Among a number of initiatives led by the Centre for Advanced Retail Studies (CARS) at Massey Business School, the Big Issues in Retail Survey is one that does not only appear to bridge the academic-practitioner gap but also provide invaluable insights into the retailing activity in New Zealand. The Big Issues in Retail Survey is an annual online survey conducted by Massey University-Auckland in collaboration with Monash University (Melbourne). The two reputable universities partnered with ACRS and Retail NZ in 2015 to bring this project to life. The first survey was run late in 2015.

The results of the 2015-16 Survey were presented at the Shop. Kiwi International Forum held mid-February in Auckland (find the survey-headline report here). The results revealed a number of interesting, and maybe controversial, findings. For instance, the portion of revenues generated by physical stores is still remarkable (60%-89%) even though the online-shopping culture in New Zealand grew significantly in the past few years. Thus, it would be interesting to learn how well the first survey performed.

This article evaluates some of the Survey’s results and compares them to what has been found and recorded since then. While the Survey covered several topics and issues, the focus here will be on respondents’ views on market changes facing retailers late in 2015. More specifically, the article looks at how respondents predicted the followings: (note that the Survey closed mid-January 2016):

  • the demand in the next three months;
  • retail sales over the next three months;
  • the number of employees over the next three months;
  • the charged prices over the next three months; and
  • the number of new stores opening in 2016.

Figure 1: Respondents’ Views on Past and Future Market Changes

Respondents were asked to rate their market views on a three-point scale with 1 being Increase, 2 being Stay The Same, and 3 being Decrease. Figure 1 depicts the results of one-sample t-tests. Respondents’ views on market changes differ significantly from the population mean (Test Value = 2). Retail sales and the charged prices were more expected to increase in the next three months than the demand, number of employees and the number of stores were. This indicates that respondents were overall optimistic about future market changes.

Figure 2: % change in the seasonally adjusted retail sales

Survey respondents predicted that the demand and retail sales would increase in the next three months. It seems that their predictions were accurate. Figure 2 presents the percentage change in the seasonally adjusted retail sales over 21 quarters from 2011 to 2016. According to Statistics NZ, the total volume and value of retail sales in March Quarter rose 0.8% and 0.6% respectively whereas in June Quarter the total volume and value of retail sales rose 2.3% and 2.2% respectively. To make it clearer, the volume of retail sales is used as a measure of demand while the value of retail sales is used as a measure of actual retail sales. The Survey’s results suggested that retailers were optimistic about future changes in the market. This optimism was seen in that 11 of the 15 industries had higher sales volumes in March Quarter while 12 of the 15 industries in June Quarter experienced higher retail sales not only in volumes but also in values (Statistics NZ).

Evidence supports respondents’ expectations of an increasing number of people hired in the following three months. According to Statistics NZ, New Zealand experienced the largest quarterly growth in labour force since December 2004 in March-2016 Quarter. Although the unemployment rate increased slightly compared to the previous quarter (December-2015 Quarter), the employment growth exceeded the population growth in March-2016 Quarter. Whereas it did not decline, the employment rate increased only by 0.2 percentage points in March-2016 Quarter. Another accurate prediction!

Figure 3: % change in CPI over year quarters from 2006 to 2016

Another market change expected by respondents was the increases in charged prices occurring over a period of three months starting from mid-January 2016. Figure 3 presents percentage change in the Consumer Price Index (CPI) over year quarters from 2006 to 2016. Over 10 years, the CPI drops significantly in December Quarters. What is interesting is to see this pattern broken one time only by a decrease in CPI in March-2015 Quarter (for more on this, visit Stats NZ). Clearly, this did not influence respondents’ views on price increases. In March-2016, prices rose 0.2% and continued to rise in the following quarter (0.4%). Respondents got this right too!

Lastly, respondents expected the number of new retail stores opening in 2016 to increase. Unfortunately, a satisfying measure of this variable has not been found. However, 2016 had witnessed new market entrants as well as developments in commercial properties. Several businesses have entered the New Zealand market for the first time and opened their stores in 2016 (e.g. Zara, H&M, Top Shop, and David Jones). In addition, many retailers shared their expansion plans and expressed their desires to open many more stores in New Zealand in the next few years. For example, Bunnings announced in February 2016 its plans to open five stores in New Zealand to add them its 50 operating stores (NewsHub.). In the grocery sector, Foodstuffs allocated (approximately) a $200 million budget for new stores and refurbishment. In response, Progressive Enterprises allocated $500 million for store expansion in New Zealand, opening 3-4 stores every year in the next few years (Euromonitor International). reported that Countdown is opening more stores than it is closing and has plans to open up to 12 supermarkets in the following three years. In the fast-food sector, Domino’s Pizza celebrated last July the opening of Store #100 and is planning to open other 100 stores across New Zealand over the next five years; the pizza chain asserts that “there is room to double its takeaway outlets in New Zealand” (NZ Harald). Perhaps this sums it all! It looks like Survey respondents had foreseen the chances for business expansion.

None of respondents’ views on future market changes was far from reality and their optimism seems to be justified. Officially published statistics and reports from multiple sources go in line, and agree, with the results, and predictions, of the Big Issues in Retail Survey 2015-16. The Survey performed particularly well in predicting changes to retail sales and prices. Thanks, of course, to the visionary leaders (respondents) who completed the survey.

I cannot wait to look at and compare the results of this year Survey which will, by the way, be presented at the Shop. Kiwi. International Forum held in Auckland this September (2017). If you have not completed the Survey, do not worry! You still have time to share your vision, opinions and the issues facing your business/es. Click here on Big Issue in Retail Survey 2016-17 and complete it. It closes Sunday 30th of April 2017.


A decision based on no information is nothing but a shot in the dark. We, humans, use our senses to register data, transform them into useful information, and then use them in decision making.

Information is power!
The absence of data is walking blindfolded….
A decision based on no information is nothing but a shot in the dark. We, humans, use our senses to register data, transform them into useful information, and then use them in decision making. When we step forward, we look in front of us to collect data and make the right decision; similarly, when we step backward, we have to turn our heads and look back to collect data and make the right decision as to where and how out foot lands. The use of data in decision making increases certainty about outcomes and reduces risks. Leading your business and staff based on ‘gut feelings’, or blindfolded, is often detrimental. Information is just as important as other marketing resources; indeed, unique information can be a competitive advantage. Companies that have more information than others lead the market, eliminate competitors, and prevent new entrants.


Many retailers cannot afford conducting market/consumer research and surveys. Besides being expensive and time-consuming, the outcome of attitudinal research and surveys is questionable.

Many retailers cannot afford conducting market/consumer research and surveys. Besides being expensive and time-consuming, the outcome of attitudinal research and surveys is questionable. With the marketing budget becoming tighter, marketers are adopting efficient marketing which Kotler (1972) defines as producing the desired customer response using the least costly marketing actions.  This seems to be increasingly shifting the marketing focus from attitudinal to behavioural research.

Although the vast majority of today’s retailers maintain customer databases at lower costs, only a few utilize these data. In current markets, customer databases are considered underutilized marketing resources which, once properly used, could well increase revenues. More retailers have recently been encouraged to invest in database marketing and consumer-base analysis. Customer data have become a significant source of revenues and help retailers furnish insights into the behaviours of their customers. It is time for retailers to leverage their existing customer data.

Retailers adopting print marketing techniques can use predictive modelling to 1) save the cost of communicating with customers unlikely to respond; 2 customize their messages/offerings; and 3) ensure that each customer receives the right catalogue, brochure, or invitation letter.

AA New Zealand customizes, and sends out, promotions to their members using members’ home addresses and the location of the petrol station at which members refill their cars most frequently.
AA New Zealand

Tesco customizes thousands of promotion packages and mails them to its 14 million UK customers.

Best Buy segmented customers who visit their stores and identified major ones. This let Best Buy:

  1. Tailor the look and feel of its stores in particular markets to fit the representation of the segments in these markets;
  2. Train its store staff to recognize customers from each segment and serve them appropriately.
Best Buy


A proper diagnosis of a business issue is necessary before ‘guessing’ the solution. One useful way to reduce risk and maximise certainty is to utilize existing data to examine the problem and test solutions prior to putting them into practice.

  • your strengths & weaknesses
  • who your business is serving
  • the behaviour of your customer
  • how to best communicate with your customers
  • how to improve your marketing productivity
  • who is loyal to your business
  • how to manage your customer relationship
Learn, learn, learn
Discover the problem!
It is perfectly normal for businesses to go through hard times experiencing stagnant sales, low response rate, higher customer churn rate, poor/weak marketing performance/stimuli, low demand for certain products, low in-store foot traffic …etc. The conventional trial-and-error approach is the wrong approach to identifying and solving problems; it could result in permanent loss of resources and serious damage to the business. A proper diagnosis of a business issue is necessary before ‘guessing’ the solution. One useful way to reduce risk and maximise certainty is to utilize existing data to examine the problem and test solutions prior to putting them into practice.

NeoRetailing utilizes available data to diagnose the problem, identifies its source, and proposes solutions that are in line with the retailer’s strategic objectives and suit its current internal capacity.

Find Opportunities!

Regardless of how well your business is currently doing, opportunities are out there waiting for someone to seize them. Unfortunately, if you do not take the initiative, a competitor will!

It is no news that the competition in the retailing sector became tougher making it harder to win a customer retain a customer, and increase the share-of-wallet. Thus, mining data for useful information is important to secure a good position in the market and consumer’s mind.  To lead in the market, a retailer should go the extra mile and fulfil the unmet needs of consumers. Data do hide a lot of useful, game-changing information!

Thomas Blischok found that the sales of diapers correlate positively and strongly with the sales of beer between 5pm and 7pm. When the two items were placed next each other in the store, the sales of both rise remarkably.

Enhance Understanding!
Consumers’ needs and wants change over time. Whether moving to a new house, having the first child, having a job promotion, or going through a period of unemployment, consumers needs/wants will not be the same as previously. As a retailer, it is important to be aware of these changes and adjust strategy and offerings accordingly.

Understanding the behaviours of consumers well enough to accurately predict it is the dream of direct marketers. This is because being able to predict customer actions rises profit significantly. In fact, recent studies found that a 1% increase in the accuracy of predicting customer actions can rise profits by 200%.

Nielsen predicts that “86% of retail growth over the next 10 years will be driven by multicultural consumers”. Consumers from different cultural backgrounds shop, spend, and respond to marketing messages differently. A well-prepared retailer will not be shocked by these differences and, instead, takes advantage, grows, and leads.