Despite the aforementioned evolution, to date mobiles continue to be predominantly used as interpersonal communication devices, enabling the share of information and the coordination of everyday life activities with family, friends, and colleagues (Ling 2004; Westlund 2007). As a particular example, empirical accounts from a Swedish study in 2007 further prove that mobile is mostly used for interpersonal communication through voice calls and SMS rather than browsing the mobile internet or accessing news information (Bohlin and Westlund, 2008; Bolin, 2008; Bolin and Westlund, 2009; Westlund, 2008a).
Some studies indicate that this behaviour is not restricted to a specific area. Katz and Aakhus (2002) concluded that mobiles are used as interpersonal communication devices in a similar way in the US, South Korea, and several European countries. By contrast, other literature state cultural differences (e.g., Campbell, 2007; Oksman and Rautiainen, 2003), which will become prevalent for the adoption and use of mobile internet services. Indeed, the uptake of mobile internet is much higher in Japan and South Korea than in Europe. According to Ofcom in the UK (2008), the average voice and data revenue per subscription shows the following trend: in Japan about one third is spent on data services, compared to 11% in Sweden. Japan has shown a high adoption of mobile internet, much thanks to the i-mode concept launched by NTT DoCoMo (Ramos et al., 2002, 2001).
Overall, there seems to be a gap between the diffusion (possible usage) and the adoption (actual usage) of mobile internet services, i.e. although many people have internet-enabled devices, they do not use them as such. However, with faster networks, multi-functional handsets and more attractive pricing, the mobile environment may soon turn into a place for developing new services beyond voice and messaging. In this context, many actors are seeking to propose and offer killer applications that can drive mobile internet adoption, among which the development of improved mobile search functionality can potentially become the disruptive factor that could transform users’ behaviour and experience.
An important aspect that has been considered in the present report is the need for data comparability. For this reason and given the heterogeneous sources of information consulted, the methodology used to analyse and forecast the mobile market evolution follows similar definitions applied by reporters and market analysts (precisely Chard, 2008, and Lane, 2008), as well as the Database of the International Telecommunication Union (ITU, 2009). This approach guarantees a certain degree of comparability, at least in terms of the order of magnitude of the forecasted figures. In the following, we will follow in the report –unless especially mentioned– the regional definition shown in Table 3.
Table 3. Segmentation of the world by regions
The term mobile internet needs some definition. In the beginning of this century, the trend was transfer web browsing to mobiles based on the WAP protocol and the Wireless Mark-up Language (WML). User acceptance was however low. There were many reasons for this (circuitswitching technology little suited to browsing, high prices, low connection quality, etc.), but more simply we can state that no attractive services and applications were offered to users. Recognizing this fact, the mobile industry started to improve the end-user experience, acknowledging that the mobile internet is not simply about repurposing the internet, but creating a relevant service and experience “made for the mobile”.
Thus, in this report we will refer mobile internet in the context of services, applications and content specifically developed for mobile users. Following the previous statement, mobile 2.0 would –by analogy to web 2.0– refer to mobile applications and services in which users have a decisive role. Without trying to aiming to provide a firm definition of mobile 2.0, the term refers to user empowerment basically through two main ways: by mobile social computing, and by integrating of the social web with the core aspects of mobility (personalization, location, context-awareness). User-generated content, services and applications that fully leverage the mobile device and the mobile context are constituents of mobile 2.0. In this context, mobile search delivers a richer mobile user experience by providing a smaller set of more relevant search results to the user based on context and location-aware capabilities. Given the need to make preliminary assumptions based on secondary sources of information (analysts and reports), this report follows the mobile 2.0 approach used by main analysts (Chard, 2008; Lane, 2008).
The estimated mobile customer base worldwide reached 3686 million users in 2008 (Figure 1). Up from 3331 million in 2007 this means approximately a 10.7% growth. During the past years the subscriber base has increased particularly in the emerging economies of Asia, Eastern Europe, Latin America and Africa, with China and India being now the largest markets (see Figure 2). In addition, countries like Indonesia, Bangladesh, Ukraine, Brazil, Mexico and Nigeria show the highest potential for growth.
Figure 1. Mobile users worldwide (2006 – 2008).
Source: ITU (2009)
It is expected that this tendency in emerging economies will continue in the following years.
The growth rate in industrialised countries (particularly in Europe with only 3% growth) is notably lower than in the emerging economies, as their markets already exceed 100% penetration in many cases (119% in EU27).
Mobile internet users reached 3,686 millions in 2008 worldwide, representing 19% of all mobile users. By regions, the main concentration is in China and India, accounting more than 50% of users worldwide. Table 4 shows global mobile and mobile internet penetration by regions.
From the previous table it is important to retain that the highest mobile internet penetration is reached in the Far East and China region (which includes Japan and South Korea), although mobile penetration there is lower compared to other regions like America or Europe.
Figure 2. Mobile users by regions (2008).
Table 4.Mobile penetration and mobile internet penetration by region (2008).
The world market for telecommunication services is estimated at 1,365 billion USD in 2008 –a 4.2% increase over the year before– and is expected to be worth over 1,416 billion USD in 2009 (Enter – IDATE, 2009). Mobile services account for 54% of the telecom services market. The estimated total turnover was 742.2 billion USD in 2008, and the annual growth rate has dropped from more than 12% in 2007 to 8% in 2008 (Figure 3).
From a regional perspective, growth rates vary greatly between mature regions and countries where growth is almost zero, and emerging regions and countries which continue to report a combined growth rate of just over 10%. Growth in North America, for instance, has been higher that in the European Union over the past three years: close to 11% in 2006 and 2007, then 5.2% in 2008 for North America; just under 5% in 2006 and 2007, then 2.7% in 2008 for the European Union. Industrialised countries in Asia have seen positive growth, although slowed down by the economic turmoil and Japanese market decline.
While the mobile customer base worldwide is still growing (12% in 2008), the global average revenue per user (ARPU) is a steady decline, dropping to 18.96 USD a month in 2008 (Figure 4). This reflects also that new customers are attracted in countries with lower incomes.
Data revenue continue to remain far behind voice revenue, still around 25% of voice revenue (Figure 5), and coming the most part of these revenues from messaging (two thirds in average). However, operators’ investments in high-speed data networks are starting to pay off, especially in South Korea and the US, where mobile operators obtain more than 50% of their data revenue from non-SMS services.
Figure 3. Global mobile services revenues (2006 - 2009). Data in billion USD.
Figure 4. Mobile average revenue per user worldwide (2006 - 2007). Data in USD.
Apart from classic voice and SMS services, mobile operators see a growth driver in data services that is indispensable for their expansion. In fact, take-off in mobile data services is becoming real with a 20% share of total data revenue worldwide in 2008 (including messaging and non-messaging revenues).
Non-messaging revenues are still low in Europe comparing to other regions, as they only represent 14% of total, with SMS being the predominant source of data revenue. According to a recent study by the EC (European Commission, 2009) most markets continue to register growth in SMS volumes which is compensating for the slight decrease in domestic prices. SMS accounts for an estimated 11% of the total mobile operators’ revenues. In contrast, all three South Korean mobile operators and the four US national mobile operators have more than 50% of their data revenues generated by non-SMS services.
Figure 5. Mobile average revenue per user: breakdown by voice and data (2006 – 2008). Data in USD.
Figure 6. Mobile subscriptions worldwide by access technology (2008).
From a regional perspective, the Far East and China region accounts for the highest penetration rate of 2G+ users (which includes 2.5G connections), followed by Europe (Figure 7).
In 2006 smartphones accounted only for 6.9% of the total market, while in 2007 this market segment reached 10.6%. This increase in sales reflects an excellent adoption of data-enabled devices. In fact, total annual sales of mobile devices reached 1,275 million units in 2008, with 71% of them sold with data facilities, of which 15% (of total sales) correspond to smartphones (Figure 8).
In Europe, 280 million units were sold in 2008, of which 19.3% were smartphones and 65.5% enhanced devices. These figures suggest that Europe is ready for data services consumption, although still behind frontrunners like Japan, with half of the mobile phones sold being smartphones in 2008 (more than 48 million). North America, with 20%, shows similar sold smartphone rates to Europe, with 20%. Although the Asian region (including India and China) showed a 17% growth from 2007, major sales correspond to basic phones, achieving 47% in 2008, and only 10% correspond to smartphones.
In terms of mobile phone manufacturers, Nokia has been leading the overall market (see data for 2008 in Figure 9), although specialised smartphone, particularly Apple, manufacturers are rapidly gaining market share.
Focusing on the smartphones market, it is important to highlight the growth of Apple and RIM devices, while the relative decline of Nokia’s market share is worrying albeit still the market leader for smartphones in 2008.
Figure 8. Global mobile handset sales by device category (2008)*.
* The categories suggested by Gartner (2008) are defined as follows:
Figure 9. World mobile handset market by manufacturers (2008).
Figure 10. Market share for smartphones by manufacturers (2006 – 2008).
Figure 11. Share of mobile internet users of mobile search services in the US by age (2008).
One proxy towards understanding the activity of mobile search users is the age structure. According to (Jupiter Research, 2008), Figure 11 estimates the average figure of mobile users using mobile search services at 8% based on the results from a survey in the US. Interestingly, the adoption is slightly higher in the young adults segment (ages 25–34; 10%) compared to younger consumers (ages 18–24; 8%), which usually represent the early adopters segment for mobile data services. In any case these figures suggest a trend that will require further research to be validated.
Other studies suggest a direct correlation between age and use of or interest in mobile services: the younger the customers, the more they use SMS and new multimedia services (Jupiter Research, 2007). Also, users aged 15–24 are more willing to accept ads on their mobile phones than are average mobile users (24% and 16%, respectively).
Even though overall adoption rates of mobile search remains in single digits, active mobile browsers are also searching from their cell phones, with 51% of daily browsers having used a familiar online search engine. These early adopters show that, similar to PC behaviour, search is often the default mechanism for navigating the mobile web. Driving more consumers to mobile browsing will likely increase the volume of searches from phones.
In line with these expectations, the “white label” mobile search provider JumpTap reports an average 100% month-over-month increase in searches across its carrier partners during 2008. Also, Google reported 50 times more searches from Apple iPhone users (on flat fee tariffs) using Google Search than from any other cell phone model, just nine months after Apple launched the iPhone.
The mobile search market is at an early stage of development. Its size is estimated at 1,480 million USD in 2008 of which 30% were generated by advertising and 70% from data traffic. The most important markets were China, West Europe and North America, with 85% of total revenues (Figure 12).
Figure 12. Mobile search revenues by region (2008) (data in million USD).
Figure 14. Traditional mobile value chain
However, the main difference in these developing markets is the opening-up of the value chain to new players that will bring dynamism to the creation of new offerings that are attractive to the end users. The mobile internet value chain (and the business relationships between players) does not follow the “linear” model any more. It is rather a complex ecosystem where stakeholders compete and interact. The ecosystem metaphor is useful to refer to a high number of players that interact within a given environment and where none of them is able to control it completely, thus collaborating and competing at the same time. The “mobile ecosystem” is currently characterised by increasingly intense mobile platform competition (Ballon, 2009a; Ramos et al., 2004) and in general terms, it can be said that the focus of the mobile industry has shifted “from single-firm revenue generation towards multi-firm control and interface issues” (Ballon, 2007).
Figure 15. Mobile internet ecosystem
Adapting the proposal of Feijóo et al. (2009b), the roles of players in any mobile content or application value network can be broadly divided into three main stages: (1) processing of information, (2) delivery, and (3) capture/use/ interaction of/with information. This three-layered structure is typical of ICT ecosystems (Fransman, 2007). As a particular mobile service, the mobile search value chain fits well within this same threelayer scheme. Figure 15 shows the main activities that players can adopt. The figure also illustrates the evolution of mobile search from the initial on-portal approach (left) to include on-device and additional input functionalities (down), the appearance of the mobile version of web search (right) and, finally, the context-aware search (up). In addition, the figure highlights (in the dark grey boxes) the activities which could be considered new and specific to mobile search.
Using the aforementioned framework and taking into account revenue models discussed in literature on mobile web, applications, content and service models (Ballon, 2007; Ballon, 2009; Bouwman, 2003; Feijóo et al., 2009b; Feijóo et al., 2009a; Rappa, 2007; Uglow, 2007), Table 5 summarizes the revenue models that mobile search providers are using or the ones they could possibly use. These revenue models are shown from the perspective of the end user and therefore; intermediate provision models (e.g., white labels, wholesale, brokerage, billing services, software development, hosting, etc) are not considered. Likewise, the revenue models for mobile operators and (hardware and software) suppliers are neither shown in the table although some of them could benefit indirectly from the adoption of mobile search. The table includes an example scenario of usage and an indication of their current existence in practice. Finally, note also that the revenue schemes presented are not exclusive and some could be complementary to each other.
This list highlights the expectations put on advertising and user profiling as main revenue streams in mobile search. In the advertising model typically the search results are provided free-of-charge to final users, and the revenues are generated from third-party advertisers. Advertising models include several very different business tactics. For instance, there could be off-portal campaigns for certain categories of services such as travel, restaurants, automotive, or consumer electronics to name a few. The traditional strategy consists of simply adding banner ads to search results, usually including a direct response method as well (a link to a microsite, a click-to-call link, or a short code). This approach fits well, for instance, for events. As another example, click-to-call text links connected to search results is a simple way to leverage the voice capabilities of mobile devices. Off-portal keyword bidding, especially for marketers offering digital content is another main example. Without exhausting all possible options, ad campaigns for product related to what mobile operators offer on their mobile portals (ringtones, games, wallpapers, music, video, etc) is an example of on-portal search. Each of these examples could be equally applied to the case of user profiling in exchange of providing mobile search results.
The list also gives a hint to the still mostly unexplored potential of value added applications where mobile search, typically of the contextaware category, is the engine within. Mobile applications providers are looking for business models to incorporate the revenue flow from the application itself, therefore departing from the traditional pay-per-download. There are different business tactics here as well. These can include time-based billing for services, event-based billing for specific situations or item-based billing as a function of the results obtained in the search.
Table 5. Main revenue models for mobile search
Forecasts of industry analysts offer valuable information on the potential of advertising and value added applications. For instance, Feijóo et al, (2009a), using data from an own survey of innovative firms in the mobile content and applications domain, provides some figures for the whole mobile domain: advertising was used by 24% of firms in the sample; pay per use /on-demand / pay-as-you-go was also a popular revenue model (17%), far outnumbering the subscription model (6%); the “secondary” revenue models of the internet (brokerage, user profiling,merchant, community, affiliation, etc) were also relatively well represented in the sample (11%).
Interestingly, some form of sharing revenues (mainly with the mobile operator) was considered by less than 1 in 2 companies in the sample (44%), and less than 1 in 4 were explicitly using “on-deck” strategies on the mobile operators’ portals (22%).
Specifically in respect to mobile search, Chard (2008) expects the advertising contribution to the total mobile search revenues to grow from 30% in 2009 to 40% in 2013. According to the Mobile Entertainment Forum (MEF), advertising revenue split ratios will likely be similar to internet with about one third for the search solution provider and about two thirds for the publisher, including as a main difference with the web, a residual percentage up to 10% for other players in the mobile value network.
Mobile search is an emerging market and its dynamism explains the notable movements that are currently taking place. New developments and promising innovative services are practically launched every week. Acquisitions, alliances and mergers are happening as small companies need scale, whereas large firms can afford the risk of buying small and innovative companies. This is expected to continue in the future, as market maturity is not at sight. The information in this section of the report is a snapshot of the mobile search market as of Spring 2009.
The categories of mobile search described in Table 2 are useful to determine the principal activities of players in the mobile search marketplace and will be used below to list some of the main players.
With increasing storage capacity, search for information stored on the mobile device gets increasingly more important. The more handsets evolve to resemble mini-computers, the more users will be overwhelmed by their capabilities but will also suffer the complexity of their devices. Poor menu navigation is already a stumbling block to buying content and accessing services. That users are aware of an available mobile data service does not mean that they know how to access it or, more importantly, install it on their device.
On-device-search is still a niche market, facing some adoption hurdles. The need for improved navigation is there, however, difficulties around distribution and business models must be addressed before it can have significant impact. Usually, software pre-installed in the terminals performs the search. Players in this market include Boopsie, Kannuu and Nuance (through acquisition of Tegic, developer of T9 predictive text, and Zi Corp).
Tegic/Nuance offers a software tool that utilises search-based navigation to enable easy access to phone data, applications, settings and services. It allows users to access them with only a few key presses instead of scrolling through the menus. Similar to shortcuts on a desktop, the software personalises the mobile phone screen with the most commonly used content and services. Additionally, the solution’s active memory remembers user data for future access and prioritises frequently used personal selections based on individual usage patterns, placing them high up on the device display.
On-portal search refers to search in closed frameworks. Often mobile companies restrict their services to their own network and applications (“walled garden” approach). “White label” search companies offer mobile operators mobile search solutions they may rebrand and provide to their customers in this manner.
The currently most common way to operate mobile search is via a separate application or a browser-based web site that seeks pages from the mobile internet. These are usually based on algorithms such as PageRank-like ones. Here, present open search solutions are internet search retrofitted for the mobile web. Search results then classified, adapted and presented in a suitable way for the terminal. For example, the number of results per page as well as the length of website titles can be adapted to the mobile terminal display size.
Meta-search engines inquire a variety of content sources and providers and “blend” and combine a multitude of search results. This allows users to search across a number of content providers through a single interface. Meta-search engines deal with technological challenges frequently encountered in the mobile space such as heterogeneity of content, and the incompatibility of devices and systems. Metasearch is likely to gain popularity with progress in mobile commerce and online stores.
Players include MCN (Mobile Content Networks) and Motricity. MCN’s customers base includes many of the major Asian/Japanese operators/providers (such as Yahoo Japan), and operators in Europe such as Telenor, as they offer a white-label search platform. Motricity gained significant traction in North America, but internal and financial issues might hamper further make its expansion. Indeed, in early 2009 the company closed down its operations outside the United States.
Searching for content within the proprietary database of a social network is technologically not dissimilar to web search. The data may be stored on a centralised or decentralised servers and data is enriched by users (e.g. tags, voting, reputation, etc.). Social classification of information (the “mobile social computing search” approach) is effective in web search and can improve search strategies also in the mobile environment, particularly in two different ways.
The first one is a specific case of contextaware search (see below), the information coming from the social network can be used to refine the search process, establishing different priorities in the way the information is presented or the results are offered. For example, if a user belongs to a social network about cars, it is expected that the search term “engine” is very likely to refer to a car engine. The second one that merits consideration as a distinctive search is the people-powered social search, using the social network to deliver not only raw information coming from a website but personal information or opinions.
Social search addresses some of the shortcomings of purely automated search. The introduction of an approach that effectively infuses human preferences and human judgments into computer algorithms gets us much closer to being able to pinpoint truly relevant information and better answers. A particular example case illustrating this is the “question and answers (Q&A)” within the group: a user poses a question expecting the response from a peer of the social network (“recommendation” engine).
Social search benefits from the increasing popularity of peer recommendation. A survey from Jupiter Research shows 64% of users will try a service or content recommended by a friend, and 69% will pass what they like along to between two and six friends. Naturally, search results would benefit from some crowd-sourcing.
As a weakness of this kind of search, the over-dependence on human involvement can limit the ability of social search sites to scale and many argue social search results can never be as comprehensive as the results from universal search engines. Another problem is what the Nielsen Norman Group calls the dilemma of “participation inequality”. As a rule, participation in the online world more or less follows a 90-91 rule, with 1% of users accounting for most of the contributions, 9% contributing from time to time and 90% of users preferring to lurk in the background rather than make a contribution. This is obviously an issue for social networks search engines across the board.
While technological challenges around scalability and algorithms might affect pure social search engines such as Hiogi, approaches combining machine-driven algorithms with social elements, can, to an extent, overcome these challenges. This is, for instance, the case of Wikia and also with Taptu, which crawls and indexes the social networking sites and destinations to expose an eclectic mix of results. In addition to aforementioned Taptu, Wikia and Hiogi, other actors include Abphone, WikiAnswers.com (coming to mobile soon with a Wiki approach), Yahoo (drawing from size, scope and strength of with Yahoo Answers community), Mahalo (a company building, and paying, contributor communities to direct searchers to relevant results), or nimble newcomers like NosyJoe (a private beta social search engine that relies on people to “sniff the Web for interesting content”).
Text-messages-based search services allow users to send a question to a central database (usually calling a special short number) and receive a reply using text based features (usually SMS), or in some cases receiving a MMS with more information, or a link to a web address where the user can find the response to their question. The response can be delivered by two ways :
The strength of text-messages-based search is its simplicity and ease of use: all mobile users are capable to send search queries via SMS. In addition, this allows mobile users without broadband access, which is still a large mass market, to search for information.
On the other hand, Schusteritsch et al. (2005) rightly pinpoint to the limitations of this interface, notably the limitation in the length of message and that the user has to know the number (to send the query to) and the message format. As a consequence, this is neither an engaging user experience nor there is room for differentiation.
Tough competition, in a buoyant and quite crowded market, does not allow for high revenue margins and forces companies in this market segment to come up with ways to purchase SMS in bulk and defray additional costs. ChaCha is market leader in the United States. The relative strength of players outside the United States is less clear. Important local provider include 4 INFO (focused on delivering alerts and advertising); MINFO (Chinese with English language service Guanxi); textperts (acquired by 118118 in the UK); Answers.com (primarily online with mobile offer), AnyQuestionAnswered (AQA), Ask and AskMeNow. Google also offers SMS search.
Progress in speech recognition paved the way carry out a search by speaking into a phone. Voicebased search fits well mobile lifestyles and the legal requirements for hands-free communication while driving, for example. Technologically speaking, voice-based search is a tougher speech recognition problem than, for instance, selecting a name from a list of contacts (voice dialling). Given the technological complexity, speech recognition functions are usually are not embedded in the device itself. Rather the speech recognition and processing runs on the server. The server receives the digitised speech through the mobile data channel for recognition. Once the question has been recognized, the search algorithm comes into operation. The results are then sent back and typically displayed as in the format of a web search text box. Alternatively the answer is sent back via SMS or “outspoken” employing a text-to-speech converter.
Systems capable of turning a query into a dialog, posing and answering questions about the request, can be foreseen in the near future. In cases of query ambiguity, narrowing the request by reducing its scope is an important part of a conversation in real-life, and so it will likely become equally important on mobile search. Despite undeniable improvement in speech recognition, user acceptance it is not clear. Users may feel more comfortable to type text rather than using voice, or vice versa.
The market leader in speech technology is Nuance Communications. Nuance has a subscription-based voice search service, Nuance Voice Control 1.0, which is been used by operators such as Sprint Nextel, Rogers Wireless, or Telus. Next integrated solution, Nuance Voice Control 2.0, is said to use both on-device and innetwork technology that includes voice search, but goes well beyond it, even allowing a user to dictate a text message.
Another interesting player is SMS search specialist ChaCha. This company uses voice recognition technology to interpret queries submitted by customer’s calling the 800 number from their mobile phones. The questions are then answered by ChaCha personnel, who conduct the internet search on the user’s behalf, and delivered to the customer’s device via SMS (see messagingservice-based search section). Recently they released a voice search service for iPhone.
Big companies like Yahoo!, Google and Microsoft are also trying to penetrating the voicebased search market. Google recently launched a voice search solution (a free Google Mobile App) to download for Apple’s iPhone. This application uses a range of commercially available and internally developed speech technologies. Google has also voice search service tailored to Android mobile phones.
Microsoft made the grade announcing a five-year deal with Verizon to use their voiceenabled web search. The speech technology has been developed internally by Microsoft and the Microsoft Research website states the objective plainly: “In the Voice Search project, we envision a future where you can ask your cell phone for any kind of information and get it”.
On its part, Yahoo! announced its “oneSearch with voice” in a keynote talk at the CTIA Wireless meeting in April 2009, using a solution from Vlingo, which in turn used speech recognition technology from IBM. Vlingo Corporation had introduced its own voice search service when it launched Vlingo for BlackBerry in June 2008. This was followed by Vlingo for iPhone in December 2008. The company plans to also launch Vlingo for Symbian and Windows Mobile devices.
The relevance of retrieved information improves the more the system is aware of the search context, i.e. takes into account the context the user is in, including the location of the user, its profile or any other useful information about the personal environment. Mobiles, being personal devices, offer valuable contextual information, because the location is (usually) known and easily identified. Using information about the user’s context, the search engine can perform a personalised search and offer tailored results. Although sometimes the participation of the user is required, these technologies are different from the “search-based on preferences”, in which an action to predefine preferences or to refine results is needed.
Mobile operators are ideally placed to develop context-aware search solutions of their own. With the limits of law, they have access to users’ location and context, profiles and purchases, the sites they browse, and the search results they consider relevant. With the adequate solutions, harnessing operator analytics and customer data, telecom operators may opt to compete with established (web) search engines and even introduce own brands. Additionally, if mobile operators chose to bolt on the mobile advertising module, they can position themselves to offer paid search advertising from a variety of advertising networks. Services making use of the terminal and user context are expected to build up in the forthcoming years; business models (e.g. personalised advertisement) will have to be adapted to it. Established “universal” search players are already developing alternative context-aware solutions. In addition to telecom operators and big search companies, there is a window of opportunity for specialised companies like Openwave to compete in this market.
A simple context-aware search application is looking for information about places or objects which located in the surroundings of the user’s (e.g. restaurants near M street in the city N). Here, the context of the query is the (present or future) location of the user; relevant information is, thus, only those contained within determined spatial perimeter. Value to this basic search would be added by matching the retrieved results with the user’s preferences. In the aforementioned example, the nearby restaurants may –for instance– be ranked by price and fitting the typical economic profile of the user. Maps and tourist information are other examples of applications where mobile search will be a key tool for the deployment of local search services.
Towards developing “fully-conscious local” applications, a number of approaches are under way to be investigated. Four distinct, but not necessarily non-overlapping, categories can be identified :
Local search is a natural fit for mobile phones. Providers of directory assistance have a paved road to get local businesses involved into mobile search and paid search advertising. This is hailed as an effective way to deliver advertising, although few brands are actually harnessing location to deliver a marketing message. Local search markets are fragmented, many players competing is small markets, making large distribution and brand awareness a big issue.
Click-to-call advertising is the major revenues, and it has generally been difficult for companies to scale up. One option could be to team up with famous search engines; but at the risk of directory assistance companies being cut out of the value chain, as it happened in the desktop internet scenario.
Different techniques are applied to audio and visual search. Using text or multimedia information as input, (different) multimedia information has to be obtained as a result. The most common way to search consists in (ideally semantically) annotating the multimedia contents with algorithms which classifies the contents. The approaches to perform the content annotation and indexing depend of the content format (video, audio, speech, images...). In addition, in a mobile environment it is necessary to adapt any multimedia search designed for PCs. For instance, in the case of searching for images on a mobile, it could be necessary to select an area of the image which contains the main interesting information for the user.
The multitude of vertical content segments (music, games, images, video) means a lot of companies playing in this space: Abphone (for images, games and music), Vtap (Veveo service for web video and embedded in a variety of devices); Truveo (also video search), Thumbplay (own tools for mobile content search), Fox Mobile (developing own applications for search and recommendation of content); plus dozens of iPhone search apps for music (overlap with offers here such as TinEye, a visual search service with focus on music).
In many cases, the content business model preserves the walled garden or silo approach, making it potentially difficult to offer users related content (images related to music downloads), or cross-sell and up-sell when the aim of the search is to find and buy content. Companies in this space must generally go direct to consumer or rely on word-of-mouth. Building a mass market reach can be difficult.
Visual search is a special case of multimedia search. The most appealing service based on visual search permit users to start a search by just snapping a photo of something with their cameraphone. A mobile search engine processes input with the help of algorithms and returns relevant digital content based on its interpretation of the user’s visual query. MMS is the usual way to send the query.
The value proposition of visual search is simple and powerful: what you see is what you get. Unfortunately, today’s services are still limited by the mobile camera’s characteristics not to able process blurry photos or intolerance to poor lighting. The search process needs to start form an image of sufficiently high quality, which is often not the case as users take casual snapshots.
In this section, we discuss the visual mobile search market as of spring 2009. It includes a landscape of the main players: mobile operators, vendors and a more detailed analysis of Nokia’s strategy.
At the March 2008 Cebit trade show in Germany, Vodafone demonstrated Otello, a service using images as input to mobile search. Users send pictures via MMS from their mobile phones and Otello returns information relevant to the picture. The technology behind the service is not disclosed, but in November 2008 Vodafone Ventures Ltd., the venture branch of Vodafone, participated in financing raised by Evolution Robotics Inc. Evolution Robotics’ technologies of visual pattern recognition and autonomous navigation are used by over 100 organisations and companies worldwide in many commercial and consumer applications.
Similarly, Japanese KDDI also offers visual search using the mobile’s camera. Its technology, ER Search, is provided by Bandai Networks and powered by Evolution Robotics’ ViPR visual pattern recognition system. As in the previous case, users can snap an object with their camera and then get content about the object sent back to the device. For example, by snapping a CD cover, the user will receive web information about the artist, sound clips of their songs and offers to download them.
In the same way, T-Mobile and Verizon subscribers can use Thrrum MMS Search (provided by 23half Inc.) to find and browse information related to their physical environment using the MMS service integrated into their phones. The publicity of the company argues that “with Thrrum, any text users see around them becomes a hyperlink that can be ‘clicked’ upon with their camera phone”. The Thrrum MMS Search beta service is available free of charge to subscribers in the United States. The MMS Search technology is protected by an extensive suite of patents pending worldwide.
Kooaba focuses on “creating hyperlinks between objects in the real world to relevant content on the mobile internet”. Founded as a spin-off company from the Federal Institute of Technology (ETH) in Zurich, the company’s technology does not rely upon artificial identifiers or tags attached to objects, but on pattern recognition, i.e. it tries “to recognise” the object. The target audience is publishers and advertising agencies, and the value proposition centres around cross-media publishing and advertising through linking print material to digital content as part of an integrated marketing strategy to increase brand awareness.
IQ Engines was founded as a collaboration of computer neuroscientists at University of California in Berkeley and in Davis. According to their website, its goal is to “bring advances in biological vision models to practical image and video search, using algorithms that are hierarchical and massively parallel (fast and accurate)”.
Mobile Acuity, a spin-off of University of Edinburgh, is focused on enabling interactive brand marketing campaigns. The flagship offer of the company is its Visual Interactivity platform. The company recently made this technology available for incorporation into applications on the Android and iPhone platforms. Services for other mobile platforms are announced. In addition to image recognition capability, Mobile Acuity’s Visual Interactivity technologies/features include Image Zoning (which determines which section of an image the consumer is pointing at and returns an appropriate response), Color ID (which analyzes the dominant colours present in an image and uses these to create a customised response), Virtual Blue-screening (which can extract the foreground of an image and reuse this within the response to the user), and Face Finder (which, in this case, extracts faces from the image for reusing them within the response returned).
SearchMe takes a different approach to visual search and lets users see what they are searching for. As users start typing, categories appear that relate to the query. Users can choose a category and see pictures of web pages that answer the search. Users can then review these pages quickly to find just the information they are looking for, before they click through. However, SearchMe has gone offline (as of November 2009) due to financial troubles. It is expected they will sell their intellectual property.
SnapNow tries to differentiate itself by including in its responses not only links to relevant web pages related to the image received, but also a number of options including a call-toaction, comparative pricing information or the chance to enter a contest. It is literally stated that the company’s patented technology “can turn any image anywhere into a URL”. SnapNow’s growing client base ranges from Woman’s Day to Manchester United to Madonna (Warner Music) to “snap-enable” their content. In the case of Madonna, SnapNow “snap-enabled” a music video, making every frame interactive and allowing consumers who capture an image to access content related to the music video on the mobile phones.
SnapTell, acquired by Amazon’s subsidiary A9.com in June 2009, is another company whose focus seems to be on enhancing print advertising, where they have a number of high-profile clients including Sports Illustrated, Rolling Stone and Martha Stewart Weddings (one of the issues is full of interactive ads that invite users to Snap.Send.Get and get tips, special offers, downloads and store locators). SnapTell claims to offer the industry’s most scalable image recognition technology for camera phones. It filed a patent (pending) for image matching said to operate with databases of millions of images, whose algorithm for image matching has been called “accumulated signed gradient”. The technology would be integrated as complete hosted mobile marketing solution available to interested companies. They also offer their technology in targeted applications for platforms such as the iPhone and Android.
Idée’s initially focused on music; yet they announced future releases that will include also books, games and support for smartphones. The service allows users to find out more about a CD by taking a picture of the cover, as well as get pricing information and music reviews. In the case of TinEye (their application for iPhone), the image identification system recognises it and delivers users links for that album on iTunes, allmusic. com, YouTube, and Wikipedia. TinEye Music does not use image metadata or watermarks.
The activities of big players in this domain are somewhat unclear. Microsoft research has taken the wraps off of a prototype visual search engine called Lincoln. No further news has been announced, as of spring 2009. Google plans are also unknown. The company made no announcement since their acquisition of mobile visual search pioneer Neven Vision in 2006. It has become also become “quiet” about Mobot, since the company was acquired by barcode and marketing company NeoMedia in January 2007.
Nokia had been talking up its visual mobile search capabilities since 2007, when they took over Pixto, a San Francisco-based start-up in the domain. In spring 2009, Nokia introduced Nokia Point & Find, in what they call a “new way to connect with information and services on the go”. A beta version of Nokia Point & Find, focusing on movies, is available in the United Kingdom and the United States and will be expanded to other countries.
Nokia Point & Find is an open service platform whereby businesses would be able to target engaging experiences and calls-to-action to consumers. The first Nokia Point & Findbased service for movies allows users to receive information on trailers, reviews, and theatres it is show by pointing the camera at a movie poster. Nokia is inviting marketing agencies and content providers to propose how their visual search service might be introduced for specific applications, campaigns or promotional activities.
Nokia’s press release states that Point & Find uses real-time image processing and recognition technologies. It uses the picture and GPS position to evaluate the object. When the object is identified, it searches a database of tagged items for associated content and services and returns a set of relevant links. For this, Nokia uses a tagging tool that it and its content, brand and advertising partners are using to build content for the service. Once tagged, the URLs for all images and related internet information that have been tagged for the service become part of the Point & Find database.
At the outset, this database will be specific to a particular service, which Nokia will be building and commercialising it one content sector at a time. But the concept is unlimited, and Nokia intents to build it out as far as it will go. Speed is said to be another asset: the content comes upon the screen immediately, as soon as the device “sees” the physical object.
It can be concluded that Point & Find is more than a mobile search tool and that Nokia has clearly taken position in visual search. It might represent a step of Nokia’s wider business strategy to bring closer together software, services and advertising.
