Guides / Building Search UI / UI & UX patterns

Voice Search with InstantSearch iOS

On this page

This guide explains how to build step by step a voice search experience using the libraries provided by Algolia. You’ll build an iOS application with a classic search bar and a button that triggers the voice input. To create this app, you’ll use the InstantSearch and Voice overlay libraries.

Building a voice search experience has three steps:

  • Input using speech-to-text
  • Fulfillment using Algolia
  • Output using speech synthesis

The speech-to-text layer - input

You must have a speech-to-text layer to convert your users’ speech into something Algolia understands (Algolia can’t process non-textual searches). You can add a speech-to-text layer in two ways:

  • Using the Chrome browser, iOS or Android native apps, or a voice platform tool like Alexa or Google Assistant with speech-to-text built-in.
  • Using a third-party service. You send the user’s speech to the service. When you receive it back, you then send it to Algolia as a search query. Some services include:

Algolia - fulfillment

In the fulfillment step, you take the users’ query and find the results in your Algolia index. You present relevant content to the user at the end of this process.

There are two parts to the Algolia fulfillment:

  • query time settings
  • index configuration Both parts are custom settings that improve search performance for the user.

Query time settings

The query time settings improve search results during query time. For instance, selecting a language for Algolia then allows you to set certain features like ignoring “noise” words that the user could enter in their search query. If you choose English as the language, and you turn on the stopwords feature, the search engine ignores words like ‘a’ and ‘an’ as they’re not relevant to the search query. This gives more exact search results.

  • Set removeStopWords and ensure to select a supported language. For example, en for English. This setting removes stop words like “a”, “an”, or “the” before running the search query.
  • Send the entire query string along as optionalWords. Speech often has words that aren’t in any of your records. With this setting, records don’t need to match all the words. Records matching more words rank higher. For example, in the spoken query “Show me all blue dresses”, only “blue dresses” may yield results for a clothing store: the other words should be optional.
  • Set ignorePlurals to true and ensure to select a supported language. For example, en for English. This setting marks words like “car” and “cars” as matching terms.
  • Apply analyticsTags to the query, including voice queries. You can activate these settings using the naturalLanguages parameter. These settings work well together when the query format is in natural language instead of keywords, for example, when your user performs a voice search.

Index configuration

Similarly, you can apply some rules related to your index. These rules are dynamic and would apply depending on what the user types in the search query. Detecting the intent of the user can help dynamically change the search results.

For example, the same search term on two different sites or apps can have a different ‘user intent’. Consider the search term ‘red’.

  • On an ecommerce website, the user wants to find products that are colored red. A rule that displays all records with a color attribute of ‘red’ will improve search results for that set of users.
  • On a movie database site, the user wants to find films that contain the phrase “red” in the title. Ensuring that the title attribute is searchable and at the top of the custom rankings will improve search results for that set of users. To learn more about how to improve search performance, see rules for dynamic filters.

Speech synthesis - output

Not all voice platforms need speech synthesis or text-to-speech. For example, a website that shows search results may be enough.

If your voice platform does need speech synthesis, your options are:

Prepare your project

To use InstantSearch iOS, you need an Algolia account. You can create a new account, or use the following credentials:

  • Application ID: latency
  • Search API Key: 927c3fe76d4b52c5a2912973f35a3077
  • Index name: STAGING_native_ecom_demo_products

These credentials give access to a preloaded dataset of products appropriate for this guide.

Create a new Xcode project

Start by creating a new Xcode project. Open Xcode, and select File -> New -> Project in the menu bar.

Project creation

Select iOS -> App template and click Next.

Project template selection

Give your application a name and click Next.

Project name input

Build and run your application (CMD + R). You should see the device simulator with a blank screen.

Simulator blank

Add project dependencies

This tutorial uses Swift Package Manager to integrate the Algolia libraries. If you prefer to use another dependency manager (Cocoapods, Carthage) please checkout the corresponding installation guides for InstantSearch and VoiceOverlay.

In the menu bar select File -> Swift Packages -> Add Package Dependency.

Spm xcode menu

Paste the GitHub link for the InstantSearch library: https://github.com/algolia/instantsearch-ios

Spm url input

Pick the latest library version on the next screen, and select the InstantSearch product from the following list:

Spm products list

Add other project dependencies in the same way:

Your dependencies are installed and you’re all set to work on your application.

Model object

Start with declaring the StoreItem model object that represents the items in the index. Add a new file StoreItem.swift to the project with the following code:

Copy 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

struct StoreItem: Codable {

  let name: String
  let brand: String?
  let description: String?
  let images: [URL]
  let price: Double?

  enum CodingKeys: String, CodingKey {
    case name
    case brand
    case description
    case images = "image_urls"
    case price
  }

  enum PriceCodingKeys: String, CodingKey {
    case value
  }

  init(from decoder: Decoder) throws {
    let container = try decoder.container(keyedBy: CodingKeys.self)
    self.name = try container.decode(String.self, forKey: .name)
    self.brand = try? container.decode(String.self, forKey: .brand)
    self.description = try? container.decode(String.self, forKey: .description)
    if let rawImages = try? container.decode([String].self, forKey: .images) {
      self.images = rawImages.compactMap(URL.init)
    } else {
      self.images = []
    }
    if
      let priceContainer = try? container.nestedContainer(keyedBy: PriceCodingKeys.self, forKey: .price),
      let price = try? priceContainer.decode(Double.self, forKey: .value) {
        self.price = price
    } else {
      self.price = .none
    }
  }

  func encode(to encoder: Encoder) throws {
    var container = encoder.container(keyedBy: CodingKeys.self)
    try container.encode(name, forKey: .name)
    try container.encode(brand, forKey: .brand)
    try container.encode(description, forKey: .description)
    try container.encode(images, forKey: .images)
    try container.encode(price, forKey: .price)
  }

}

Result views

Add a file ProductTableViewCell.swift for visually displaying the store item in the results list.

Copy 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

import Foundation
import UIKit
import SDWebImage

class ProductTableViewCell: UITableViewCell {
  
  let itemImageView: UIImageView
  let titleLabel: UILabel
  let subtitleLabel: UILabel
  let priceLabel: UILabel

  let mainStackView: UIStackView
  let labelsStackView: UIStackView
  
  override init(style: UITableViewCell.CellStyle, reuseIdentifier: String?) {
    itemImageView = .init()
    titleLabel = .init()
    subtitleLabel = .init()
    mainStackView = .init()
    labelsStackView = .init()
    priceLabel = .init()
    super.init(style: style, reuseIdentifier: reuseIdentifier)
    layout()
    backgroundColor = .white
  }
  
  required init?(coder: NSCoder) {
    fatalError("init(coder:) has not been implemented")
  }
  
  private func layout() {
    itemImageView.sd_imageIndicator = SDWebImageActivityIndicator.gray
    itemImageView.translatesAutoresizingMaskIntoConstraints = false
    itemImageView.clipsToBounds = true
    itemImageView.contentMode = .scaleAspectFit
    itemImageView.layer.masksToBounds = true

    titleLabel.translatesAutoresizingMaskIntoConstraints = false
    titleLabel.font = .systemFont(ofSize: 15, weight: .bold)
    titleLabel.numberOfLines = 1
    
    subtitleLabel.translatesAutoresizingMaskIntoConstraints = false
    subtitleLabel.font = .systemFont(ofSize: 13, weight: .regular)
    subtitleLabel.textColor = .gray
    subtitleLabel.numberOfLines = 1
    
    priceLabel.translatesAutoresizingMaskIntoConstraints = false
    priceLabel.font = .systemFont(ofSize: 14)
        
    labelsStackView.axis = .vertical
    labelsStackView.translatesAutoresizingMaskIntoConstraints = false
    labelsStackView.spacing = 3
    labelsStackView.addArrangedSubview(titleLabel)
    labelsStackView.addArrangedSubview(subtitleLabel)
    labelsStackView.addArrangedSubview(priceLabel)
    labelsStackView.addArrangedSubview(UIView())
    
    mainStackView.axis = .horizontal
    mainStackView.translatesAutoresizingMaskIntoConstraints = false
    mainStackView.spacing = 20
    mainStackView.addArrangedSubview(itemImageView)
    mainStackView.addArrangedSubview(labelsStackView)
    
    contentView.addSubview(mainStackView)
    contentView.layoutMargins = .init(top: 5, left: 3, bottom: 5, right: 3)

    mainStackView.pin(to: contentView.layoutMarginsGuide)
    itemImageView.widthAnchor.constraint(equalTo: itemImageView.heightAnchor).isActive = true
  }
  
}

Define a ProductTableViewCell extension. Its setup method configures a cell with a StoreItem instance:

Copy 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

extension ProductTableViewCell {
  
  func setup(with productHit: Hit<StoreItem>) {
    let product = productHit.object
    itemImageView.sd_setImage(with: product.images.first)
    
    if let highlightedName = productHit.hightlightedString(forKey: "name") {
      titleLabel.attributedText = NSAttributedString(highlightedString: highlightedName,
                                                     attributes: [
                                                      .foregroundColor: UIColor.tintColor])
    } else {
      titleLabel.text = product.name
    }
    
    if let highlightedDescription = productHit.hightlightedString(forKey: "brand") {
      subtitleLabel.attributedText = NSAttributedString(highlightedString: highlightedDescription,
                                                        attributes: [
                                                          .foregroundColor: UIColor.tintColor
                                                        ])
    } else {
      subtitleLabel.text = product.brand
    }
    
    if let price = product.price {
      priceLabel.text = "\(price) €"
    }
    
  }

}

Results view controller

Algolia doesn’t provide a ready-to-use results view controller, but you can create one with the tools in the InstantSearch library by copying and pasting the following code to your project.

Read more about Hits in the API reference.

Add a StoreItemsTableViewController class, which implements the HitsController protocol. This view controller presents the search results with the previously declared ProductTableViewCell.

Copy 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

import Foundation
import UIKit
import InstantSearch

class StoreItemsTableViewController: UITableViewController, HitsController {
  
  var hitsSource: HitsInteractor<Hit<StoreItem>>?
  
  var didSelect: ((Hit<StoreItem>) -> Void)?
  
  let cellIdentifier = "cellID"
  
  override func viewDidLoad() {
    super.viewDidLoad()
    tableView.register(ProductTableViewCell.self, forCellReuseIdentifier: cellIdentifier)
  }
  
  override func tableView(_ tableView: UITableView, numberOfRowsInSection section: Int) -> Int {
    return hitsSource?.numberOfHits() ?? 0
  }
  
  override func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
    guard let cell = tableView.dequeueReusableCell(withIdentifier: cellIdentifier, for: indexPath) as? ProductTableViewCell else {
      return UITableViewCell()
    }
    guard let hit = hitsSource?.hit(atIndex: indexPath.row) else {
      return cell
    }
    cell.setup(with: hit)
    return cell
  }
  
  override func tableView(_ tableView: UITableView, heightForRowAt indexPath: IndexPath) -> CGFloat {
    return 80
  }
  
  override func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) {
    if let hit = hitsSource?.hit(atIndex: indexPath.row) {
      didSelect?(hit)
    }
  }
  
}

Create a basic search experience

All the auxiliary parts of the application are ready. You can now set up the main view controller of the application. In your Xcode project, open the ViewController.swift file and import the InstantSearch library.

Copy 
1
2
3
4
5
6
7
8
9
10
11

import UIKit
import InstantSearch

class ViewController: UIViewController {

  override func viewDidLoad() {
    super.viewDidLoad()
    // Do any additional setup after loading the view.
  }

}

Start by creating a classic search interface with a search bar and a results list. Fill your ViewController class with a minimal set of InstantSearch components for a basic search experience.

  • HitsSearcher: component that performs search requests and handles search responses.
  • UISearchController: view controller that manages the display of search results based on interactions with a search bar. UIKit component.
  • TextFieldController: controller that binds the SearchBar with other InstantSearch components.
  • SearchBoxConnector: connector that encapsulates the textual query input handling logic and connects it with HitsSearcher and TextFieldController.
  • StoreItemsTableViewController: controller that presents the list of search results.
  • HitsConnector: connector that encapsulates the search hits handling logic and connects it with HitsSearcher and HitsController.

Your ViewController class should look as follows:

Copy 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

import UIKit
import InstantSearch
import InstantSearchVoiceOverlay

class ViewController: UIViewController {
  
  let searchController: UISearchController
  let searcher: HitsSearcher
  
  let searchBoxConnector: SearchBoxConnector
  let textFieldController: TextFieldController
  
  let hitsConnector: HitsConnector<Hit<StoreItem>>
  let searchResultsController: StoreItemsTableViewController
    
  override init(nibName nibNameOrNil: String?, bundle nibBundleOrNil: Bundle?) {
    searcher = .init(client: .newDemo,
                     indexName: Index.Ecommerce.products)
    searchResultsController = .init()
    hitsConnector = .init(searcher: searcher,
                          controller: searchResultsController)
    searchController = .init(searchResultsController: searchResultsController)
    textFieldController = .init(searchBar: searchController.searchBar)
    searchBoxConnector = .init(searcher: searcher,
                               controller: textFieldController)
    super.init(nibName: nibNameOrNil, bundle: nibBundleOrNil)
  }
  
  required init?(coder: NSCoder) {
    fatalError("init(coder:) has not been implemented")
  }
  
  override func viewDidLoad() {
    super.viewDidLoad()
    // Do any additional setup after loading the view.
  }
  
}

Add the private setup method which configures the viewController and its searchController. Call it from the viewDidLoad method. Next, make the searchController active in the viewDidAppear method to make the search appear on each appearance of the main view.

Copy 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

import UIKit
import InstantSearch
import InstantSearchVoiceOverlay

class ViewController: UIViewController {
  
  let searchController: UISearchController
  let searcher: HitsSearcher
  
  let searchBoxConnector: SearchBoxConnector
  let textFieldController: TextFieldController
  
  let hitsConnector: HitsConnector<Hit<StoreItem>>
  let searchResultsController: StoreItemsTableViewController
    
  override init(nibName nibNameOrNil: String?, bundle nibBundleOrNil: Bundle?) {
    searcher = .init(client: .newDemo,
                     indexName: Index.Ecommerce.products)
    searchResultsController = .init()
    hitsConnector = .init(searcher: searcher,
                          controller: searchResultsController)
    searchController = .init(searchResultsController: searchResultsController)
    textFieldController = .init(searchBar: searchController.searchBar)
    searchBoxConnector = .init(searcher: searcher,
                               controller: textFieldController)
    super.init(nibName: nibNameOrNil, bundle: nibBundleOrNil)
  }
  
  required init?(coder: NSCoder) {
    fatalError("init(coder:) has not been implemented")
  }
  
  override func viewDidLoad() {
    super.viewDidLoad()
    setup()
  }
  
  override func viewDidAppear(_ animated: Bool) {
    super.viewDidAppear(animated)
    searchController.isActive = true
  }
  
  private func setup() {
    title = "Voice Search"
    view.backgroundColor = .white
    navigationItem.searchController = searchController
    searchController.hidesNavigationBarDuringPresentation = false
    searchController.showsSearchResultsController = true
    searchController.automaticallyShowsCancelButton = false
    searcher.search()
  }
  
}

Embed the main view controller in the navigation controller. Open the Main storyboard file. Select the view in the View Controller Scene. In the Xcode menu select Editor > Embed In > Navigation Controller.

Xcode embed in nav controller

Build and run your application. The basic search experience is ready: you can type your search query and get instant results.

Simulator basic search

Create a voice search experience

This is a two-step process:

  1. Prepare the project for voice input and speech recognition.
  2. Add a button on the right of the search bar that triggers the voice input.

Setup permission request

By default, the VoiceOverlay library uses the AVFoundation framework for voice capturing and the Speech framework for speech to text transformation. Both libraries come with the iOS SDK. These frameworks require the microphone and speech recognition permissions, respectively, from the operating system. The VoiceOverlay library takes care of the permission request logic and appearance, all you have to do is to provide the reason you need these permissions in the info.plist file .

Open the info.plist file of your VoiceSearch target in the Xcode editor, and add the following keys:

  • Privacy - Microphone Usage Description
  • Privacy - Speech Recognition Usage Description with values : Voice input.

In the end your info.plist should look as follows:

Xcode info plist

Add voice input logic

First, add import InstantSearchVoiceOverlay at the top of your ViewController.swift file.

Copy 
1
2
3

import UIKit
import InstantSearch
import InstantSearchVoiceOverlay

Declare VoiceOverlayController in the ViewController:

Copy 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

class ViewController: UIViewController {

  let searchController: UISearchController
  let searcher: HitsSearcher
  
  let searchBoxConnector: SearchBoxConnector
  let textFieldController: TextFieldController
  
  let hitsConnector: HitsConnector<Hit<StoreItem>>
  let searchResultsController: StoreItemsTableViewController
  
  let voiceOverlayController: VoiceOverlayController

  ...
}

Add a private method which handles the presentation of errors.

Copy 
1
2
3
4
5
6
7
8
9
10
11

private func present(_ error: Error) {
  let alertController = UIAlertController(title: "Error",
                                          message: error.localizedDescription,
                                          preferredStyle: .alert)
  alertController.addAction(.init(title: "OK",
                                  style: .cancel,
                                  handler: .none))
  navigationController?.present(alertController,
                                animated: true,
                                completion: nil)
}

Implement the searchBarBookmarkButtonClicked function of the UISearchBarDelegate protocol in the extension of the view controller. This function binds the voice input callback to SearchBoxInteractor, encapsulated by the SearchConnector in your class declaration.

Copy 
1
2
3
4
5
6
7
8
9
10
11
12
13
14

extension ViewController: UISearchBarDelegate {
  
  func searchBarBookmarkButtonClicked(_ searchBar: UISearchBar) {
    voiceOverlayController.start(on: self.navigationController!) { [weak self] (text, isFinal, _) in
      self?.searchBoxConnector.interactor.query = text
    } errorHandler: { error in
      guard let error = error else { return }
      DispatchQueue.main.async { [weak self] in
        self?.present(error)
      }
    }
  }
  
}

Customize the search bar bookmark button in the setup method. Set the view controller as a delegate of the search bar.

Copy 
1
2
3
4
5
6
7
8
9
10
11
12

private func setup() {
  title = "Voice Search"
  view.backgroundColor = .white
  navigationItem.searchController = searchController
  searchController.hidesNavigationBarDuringPresentation = false
  searchController.showsSearchResultsController = true
  searchController.automaticallyShowsCancelButton = false
  searchController.searchBar.setImage(UIImage(systemName: "mic.fill"), for: .bookmark, state: .normal)
  searchController.searchBar.showsBookmarkButton = true
  searchController.searchBar.delegate = self
  searcher.search()
}

In the end, the code of your ViewController should look as follows:

Copy 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

import UIKit
import InstantSearch
import InstantSearchVoiceOverlay

class ViewController: UIViewController {
  
  let searchController: UISearchController
  let searcher: HitsSearcher
  
  let searchBoxConnector: SearchBoxConnector
  let textFieldController: TextFieldController
  
  let hitsConnector: HitsConnector<Hit<StoreItem>>
  let searchResultsController: StoreItemsTableViewController
  
  let voiceOverlayController: VoiceOverlayController
  
  override init(nibName nibNameOrNil: String?, bundle nibBundleOrNil: Bundle?) {
    searcher = .init(client: .newDemo,
                     indexName: Index.Ecommerce.products)
    searchResultsController = .init()
    hitsConnector = .init(searcher: searcher,
                          controller: searchResultsController)
    searchController = .init(searchResultsController: searchResultsController)
    textFieldController = .init(searchBar: searchController.searchBar)
    searchBoxConnector = .init(searcher: searcher,
                               controller: textFieldController)
    voiceOverlayController = .init()
    super.init(nibName: nibNameOrNil, bundle: nibBundleOrNil)
  }
  
  required init?(coder: NSCoder) {
    fatalError("init(coder:) has not been implemented")
  }
  
  override func viewDidLoad() {
    super.viewDidLoad()
    setup()
  }
  
  override func viewDidAppear(_ animated: Bool) {
    super.viewDidAppear(animated)
    searchController.isActive = true
  }
  
  private func setup() {
    title = "Voice Search"
    view.backgroundColor = .white
    navigationItem.searchController = searchController
    searchController.hidesNavigationBarDuringPresentation = false
    searchController.showsSearchResultsController = true
    searchController.automaticallyShowsCancelButton = false
    searchController.searchBar.setImage(UIImage(systemName: "mic.fill"), for: .bookmark, state: .normal)
    searchController.searchBar.showsBookmarkButton = true
    searchController.searchBar.delegate = self
    searcher.search()
  }
  
  private func present(_ error: Error) {
    let alertController = UIAlertController(title: "Error",
                                            message: error.localizedDescription,
                                            preferredStyle: .alert)
    alertController.addAction(.init(title: "OK",
                                    style: .cancel,
                                    handler: .none))
    navigationController?.present(alertController,
                                  animated: true,
                                  completion: nil)
  }
  
}

extension ViewController: UISearchBarDelegate {
  
  func searchBarBookmarkButtonClicked(_ searchBar: UISearchBar) {
    voiceOverlayController.start(on: self.navigationController!) { [weak self] (text, isFinal, _) in
      self?.searchBoxConnector.interactor.query = text
    } errorHandler: { error in
      guard let error = error else { return }
      DispatchQueue.main.async { [weak self] in
        self?.present(error)
      }
    }
  }
  
}

To test your voice search, build and run your application. You should see the voice input button on the right of the search bar.

Simulator voice search

The VoiceOverlay should appear when you tap the voice input button. At the first launch, it asks for the permissions mentioned in the setup permissions request section.

Simulator permission request

Simulator os permission request

Once you give all the authorizations, the voice input interface appears. Try to say something and get the instant search results.

Simulator voice input

Simulator voice result

You can find the complete source code in the Examples section of the InstantSearch iOS repository.

Conclusion

With a few components and Algolia’s libraries, you can build a voice search experience for your iOS application. You can customize your search experience and make it unique by modifying InstantSearch components, as well as the VoiceOverlay components.

Previous
Facet Display
Next
Conditional display
Did you find this page helpful?
InstantSearch iOS v7