Browsing articles in "Blog"

Working with PDF forms on Android

Jul 18, 2015   //   by Theo   //   Blog  //  No Comments

Background

Traditionally, working with PDF on Android has been hard because there is no PDF support built into the OS. To be able to work with PDF across the major Android OS versions in use, you typically have to resort to third-party commercial libraries or online services that can process PDF files remotely. Both approaches have their problems: commercial PDF libraries require thousands of dollars in yearly maintenance while online services introduce significant delays in processing, plus don’t work when Internet is not available.

Recent Developments

Recently, some new options have emerged that make working with PDF on Android much more convenient. A new project called  PDFBox-Android brings the power of the well-known Apache PDFBox open-source library to Android. You can now create, alter, merge, split and render PDF documents easily and for free. The main advantages of this solution are:

  • Open source license based on Apache 2 that allows you to use the component in commercial applications without onerous license fees
  • Pure Java library that does not require separate C/C++ libraries for different architectures
  • Large community of Java developers already using this library for the desktop. That means that there are numerous StackOverflow threads and blog posts to help you with common usage scenarios.

The Android OS has also added some PDF functionality in recent versions. In API level 19 Google introduced a new class called PdfDocument to generate PDF documents from native content, and in API level 21 Google added another class called PdfRenderer to allow rendering a PDF document. However, the usefulness of these options is limited because they are not available in older Android versions currently in use.

Filling a PDF Form

In this post we will use PDFBox-Android to fill a PDF form. We will use an AcroForms PDF with pre-defined textbox and checkbox fields. In the next part of this series, we will look at inserting a signature into the PDF.

The actual form we will use in this blog post can be found here. If you want to create your own small forms with AcroForm fields, you can use the free PDFescape online form editor. When working professionally with large forms that have a lot of fields, you should probably use Adobe Acrobat because it has automatic AcroForm field insertion can save you from the drudgery of manually defining AcroForm fields.

Adding PDFBox-Android to your Project

If you are using Gradle and Android Studio, you can simply add the jcenter repository and a reference to PDFBox-Android 1.8.9.0 (latest version as of now), as shown below:

In your project’s build.gradle:

allprojects {
    repositories {
        jcenter()
    }
}

In your app’s build.gradle:

dependencies {
 compile 'org.apache:pdfbox-android:1.8.9.0'
}

Filling the AcroForm Fields

To get started, we’ll create a skeleton function called fillPdf that opens a PDF, retrieves the AcroForm, makes changes to fields and then closes and save the changes to another PDF file.

public fillPdf(String inFileName, String outFileName)
{
    try
    {
        // Load the document
        document = PDDocument.load(inFileName);

        // Get the AcroForm
        PDDocumentCatalog docCatalog = pdfDocument.getDocumentCatalog();
        PDAcroForm acroForm = docCatalog.getAcroForm();

        // TODO: Set fields 

        // Save document
        document.save(outFileName);

        // Close document
        document.close();
    }
    catch (IOException ioe)
    {
        ioe.printStackTrace();
    }
}

We change the fields using the setTextField and setCheckBox functions whose implementation is shown below:

public void setTextField(PDAcroForm acroForm, String textFieldName, String text)
{
    PDFieldTreeNode treeNode = acroForm.getField(textFieldName);
    if( treeNode != null )
    {
        treeNode.setValue(text);
    }
    else
    {
        Log.e(TAG, "No text field found with name:" + name);
    }
}

public void setCheckBoxField(PDAcroForm acroForm, String checkBoxFieldName, boolean isChecked)
{
    PDFieldTreeNode treeNode = acroForm.getField(checkBoxFieldName);
    if( treeNode != null && treeNode instanceof PDCheckbox)
    {
        if (isChecked)
        {
            ((PDCheckBox)treeNode).check();
        }
        else
        {
            ((PDCheckBox)treeNode).unCheck();
        }
    }
    else
    {
        Log.e(TAG, "No checkbox field found with name:" + name);
    }
}

Finally, we will need to add the calls to edit the form fields in the fillPdf function, replacing the TODO. The final code for fillPdf is shown below.

public fillPdf(String inFileName, String outFileName)
{
    try
    {
        // Load the document
        document = PDDocument.load(inFileName);

        // Get the AcroForm
        PDDocumentCatalog docCatalog = pdfDocument.getDocumentCatalog();
        PDAcroForm acroForm = docCatalog.getAcroForm();

        // Set AcroForm fields
        setTextField(acroForm, "name", "John Doe");
        setCheckboxField(acroForm, "male", true);
        setTextField(acroForm, "address", "123 Broadway");
        setTextField(acroForm, "city", "New York");
        setTextField(acroForm, "state", "NY");
        setTextField(acroForm, "zip", "10001");

        // Save document
        document.save(outFileName);

        // Close document
        document.close();
    }
    catch (IOException ioe)
    {
        ioe.printStackTrace();
    }
}

Stay tuned for the next part

In the next part we will look at adding the signature into the PDF document. Until then, let us know of any questions in the comments section.

State Preservation in Backstack Fragments

Jun 28, 2015   //   by Theo   //   Blog  //  No Comments

Background

When adding and restoring fragments to the backstack, additional care must be taken that the fragment lifecycle is properly handled, so instance state preservation works correctly. This post looks at a couple of gotchas involving state preservation and backstack fragments.

Gotcha #1: Instance state not restored when returning from backstack

Consider the following code for a fragment which displays a list of countries received from the server. It overrides onSaveInstanceState to save its state (the list of countries) and onCreateView to restore its state.

private List<String> mCountries;

@Override
public View onCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState)
{
    View view = inflater.inflate(R.layout.fragment_countries, container, false);

    if (savedInstanceState == null)
    {
        // Populate countries by calling AsyncTask
    }
    else
    {
       // Populate countries by extracting them from saved instance state bundle
    }

    return view;
}

public void onSaveInstanceState(Bundle outState)
{
    // Save countries into bundle
}

While testing this code, we note that something is not quite right with instance state restore in the following scenario:

  1. Another fragment moves to the foreground and the countries fragment is added to the backstack
  2. We press the back button to move the countries fragment from the backstack to the foreground
  3. The countries fragment reloads the data from the server instead of using the saved instance state data

We would like to avoid reloading data from the server when the fragment is restored from the backstack because that creates additional traffic on the webserver, and causes a 1-2 second delay in the UI while the data is loaded.

So why does this happen? To find out, we will need to take a close look at the fragment lifecycle.

With the regular fragment lifecycle used during fragment recreation, both the view and the fragment are destroyed, meaning that onSaveInstanceState, onDestroyView and then onDestroy are called (among others). During re-creation, both the fragment and the view are re-created, so onCreate and onCreateView are called. Both provide the instance state saved before destruction.

fragment_recreation

On the other hand, with the backstack fragment lifecycle, when the fragment is added to the backstack only the view is destroyed, while the rest of the fragment is preserved on the backstack with its member variables intact. This means that only onDestroyView is called, while onSaveInstanceState and onDestroy are not. When the fragment is brought back to the foreground, only the view has to be re-created, so onCreateView is called. Since onSaveInstanceState was not called, the saved instance state in onCreateView is null.

fragment_backstack

This is the root cause why our code is not working correctly. Our code above assumes that a null instance state means that the data is not loaded, so it reloads the data from server. Clearly this is not appropriate for the backstack case.

So how do we fix this? One easy way is to stop using the instance state as a proxy for whether the data is loaded, and just check the data itself. This means that instead of checking whether savedInstanceState is null, we will check whether mCountries is null. The revised code is shown below:

private List<String> mCountries;

@Override
public View onCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState)
{
    View view = inflater.inflate(R.layout.fragment_countries, container, false);

    if (mCountries == null) // Updated check
    {
        // Populate countries by calling AsyncTask
    }
    else
    {
        // Populate countries by extracting them from saved instance state bundle
    }

    return view;
}

public void onSaveInstanceState(Bundle outState)
{
 // Save countries into bundle
}

The key takeaway here is that you should directly check the state of the fragment rather than the saved instance state, since a null instance state in onCreateView does not necessarily mean that the state is uninitialized.

Gotcha #2: Instance state not restored when rotating the device twice

So we’re pretty satisfied with the fix, but while testing the code above we come across a strange scenario involving rotation.

  1. Another fragment moves to the foreground and the countries fragment is added to the backstack
  2. We rotate the device twice, while the other fragment is in the foreground
  3. We press the back button to restore the countries fragment from the backstack
  4. The countries fragment reloads the data from the server, instead of using the saved instance state data

What’s going on here? All other scenarios seem to work properly.

Once again, understanding the root cause for this behavior requires a deep dive into the fragment lifecycle. This time we will look at how the regular fragment lifecycle and the backstack fragment lifecycle interact.

  1. When the fragment is added to the backstack, we pass through the events shown above in the first part of Figure 2. The view is destroyed and the fragment is added to the backstack with its member variables intact. onDestroyView is called, but onSaveInstanceState and onDestroy are not.
  2. When the device is rotated the first time, the entire activity that contains the fragments has to be re-created. That means that all the fragments found on the backstack have to be re-created too. During tear-down, our countries fragment receives the onSaveInstanceState and onDestroy events. Then during re-creation, the fragment is re-created, but because the fragment is on the backstack, the view is not. That means that the onCreate event is called, but onCreateView is not. Without onCreateView, the mCountries member variable is not populated.
  3. When the device is rotated a second time, onSaveInstanceState and onDestroy are called. Since mCountries has not been populated from the previous instance and is null, the saved instance state will now contain null for the countries data.
  4. When the countries fragment is restored from the backstack, onCreate and onCreateView are called. For both events the countries data is null, so the countries data will not be restored.

So how do we fix this? Obviously, restoring the state in onCreateView is the main reason why the instance state is lost, since that event will not be called when the fragment is on the backstack. One easy fix is to restore our instance state in the fragment’s onCreate instead of onCreateView. The revised code is shown below:

private List<String> mCountries;</pre>
@Override
public void onCreate(Bundle savedInstanceState)
{
    if (savedInstanceState != null)
    {
        // Populate countries from bundle
    }
}

@Override
public View onCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState)
{
    View view = inflater.inflate(R.layout.fragment_countries, container, false);

    if (mCountries == null)
    {
        // Populate countries by calling AsyncTask
    }

    return view;
}

public void onSaveInstanceState(Bundle outState)
{
    // Save countries into bundle
}

The key takeaway here is that the instance state should be restored in onCreate rather than onCreateView, since onCreate is always called on fragment creation, while onCreateView is not necessarily.

Conclusion

Hopefully this article was helpful in dealing with instance state and backstack fragments in your apps. Let us know about any questions or suggestions in the comments.

Which HTML rendering engine does the JellyBean WebView use?

Apr 25, 2013   //   by Theo   //   Blog  //  2 Comments

Background

We often work with hybrid Android apps that have both native code (written in Java) and web code (written in HTML /JavaScript). When we do, we often need a quick way to test whether the HTML component works properly. Typically, this means loading the HTML in a mobile browser. If the HTML loads successfully and renders correctly, then the HTML should also work in the app’s webview.

When we do this kind of testing, we try to make sure that the web browser employed for testing uses the same HTML redering engine as the webview. On Android versions up to JellyBean, this means using the classic Android browser, which according to Google shares the same rendering engine as the webview. On JellyBean though, with Google’s introduction of the Chrome browser and with some talk about a Chrome-based webview, things got a bit murky… Does the JellyBean webview use the Android Browser or the Google Chrome rendering engine? Which browser should you use to test your HTML on JellyBean?

The Search

To get a definitive answer to this question we are going to dive into the Android codebase. For that, we will turn to GrepCode, a site that lets anyone quickly browse multiple versions of the Android OS codebase without needing to download the multi-gigabyte source code repositories.

We begin our search in the source code for Android 4.2.2 r1, which can be found here.

Since we’re interested in the WebView class, let’s find that class in the source code. We know from the Android documentation that the WebView class is located in the android.webkit package. You can browse to that package and open the WebView.java file.

Any Java object is instantiated using its constructor, so let’s start there. There are several chained constructors in the WebView source code. All of them eventually call this constructor at line 501:

@SuppressWarnings("deprecation") // for super() call into deprecated base class constructor.
protected WebView(Context context, AttributeSet attrs, int defStyle,
        Map<String, Object> javaScriptInterfaces, boolean privateBrowsing) {
    super(context, attrs, defStyle);
    if (context == null) {
        throw new IllegalArgumentException("Invalid context argument");
    }
    checkThread();

    ensureProviderCreated();
    mProvider.init(javaScriptInterfaces, privateBrowsing);
}

The checkThread function is not too interesting, so let’s look at the next one called ensureProviderCreated:

private void ensureProviderCreated() {
    checkThread();
    if (mProvider == null) {
        // As this can get called during the base class constructor chain, pass the minimum
        // number of dependencies here; the rest are deferred to init().
        mProvider = getFactory().createWebView(this, new PrivateAccess());
    }
}

Based on the code above, the provider is generated using a Factory pattern. The returned WebViewFactoryProvider class is instantiated in the getFactory function:

private static synchronized WebViewFactoryProvider getFactory() {
    // For now the main purpose of this function (and the factory abstration) is to keep
    // us honest and minimize usage of WebViewClassic internals when binding the proxy.
    checkThread();
    return WebViewFactory.getProvider();
}

Notice that most of the work in the function above is done in a static function called WebViewFactory.getProvider. Clicking on getProvider in GrepCode will jump us the WebViewFactory.java file. Within that function, at line 52, we see:

    // For debug builds, we allow a system property to specify that we should use the
    // Chromium powered WebView. This enables us to switch between implementations
    // at runtime. For user (release) builds, don't allow this.
    if (Build.IS_DEBUGGABLE && SystemProperties.getBoolean("webview.use_chromium", false)) {
        StrictMode.ThreadPolicy oldPolicy = StrictMode.allowThreadDiskReads();
        try {
            sProviderInstance = loadChromiumProvider();
            if (DEBUG) Log.v(LOGTAG, "Loaded Chromium provider: " + sProviderInstance);
        } finally {
            StrictMode.setThreadPolicy(oldPolicy);
        }
    }

    if (sProviderInstance == null) {
        if (DEBUG) Log.v(LOGTAG, "Falling back to default provider: "
            + DEFAULT_WEBVIEW_FACTORY);
        sProviderInstance = getFactoryByName(DEFAULT_WEBVIEW_FACTORY,
            WebViewFactory.class.getClassLoader());
        if (sProviderInstance == null) {
            if (DEBUG) Log.v(LOGTAG, "Falling back to explicit linkage");
            sProviderInstance = new WebViewClassic.Factory();
        }
    }

Within the same file, at line 32, we have:

private static final String DEFAULT_WEBVIEW_FACTORY = "android.webkit.WebViewClassic$Factory";
private static final String CHROMIUM_WEBVIEW_FACTORY = "com.android.webviewchromium.WebViewChromiumFactoryProvider";

As you can see from these last two code snippets, the default WebView provider is still the “classic” webview that is based on the Android Browser rendering engine. Apparently there is work underway to switch the webview to the Chrome rendering engine, but right now that’s only available in debug Android builds using a special flag.

Conclusion

Based on an examination of the Android source code, the webview does not use the Chrome rendering engine in Android 4.2 (JellyBean). The webview is still powered by the Android Browser rendering engine.

Copying EXIF metadata using Sanselan

Dec 8, 2012   //   by Theo   //   Blog  //  1 Comment

The Challenge

After resizing, rotating, or otherwise modifing an image, you typically want to retain all the EXIF metadata from the original image. Unfortunately the EXIF metadata is not automatically copied for you, so you need to explicitly copy it from the original.

Review of Existing Solutions

There are several solutions that currently address this, but most of them fall short in one way or another.

Some solutions attempt to copy the data using Android’s built-in ExifInterface. Unfortunately,  ExifInterface has EXIF data corruption issues on some Android OS versions. In addition, ExifInterface is only able to read and write a limited number of EXIF fields, meaning that these solutions are not able to preserve all EXIF fields.

Better solutions are based on the Sanselan Android library, which is a superior choice for writing EXIF data. Unfortunately the one solution we have seen that uses Sanselan does not work as intended: in addition to copying the EXIF metadata it also overwrites the image data, effectively undoing the image transformation work.
http://kb.trisugar.com/node/22096

Our Solution

Our solution extends the Sanselan solution above. To work around an apparent bug in Sanselan that others have noted, we do not apply the previous EXIF data set to the new image. Instead we create a new EXIF data set, loop through existing fields in the old data set and copy each old field to the new data set.

public static boolean copyExifData(File sourceFile, File destFile, List<TagInfo> excludedFields)
{
    String tempFileName = destFile.getAbsolutePath() + ".tmp";
    File tempFile = null;
    OutputStream tempStream = null;

    try
    {
        tempFile = new File (tempFileName);

        TiffOutputSet sourceSet = getSanselanOutputSet(sourceFile, TiffConstants.DEFAULT_TIFF_BYTE_ORDER);
        TiffOutputSet destSet = getSanselanOutputSet(destFile, sourceSet.byteOrder);

        // If the EXIF data endianess of the source and destination files
        // differ then fail. This only happens if the source and
        // destination images were created on different devices. It's
        // technically possible to copy this data by changing the byte
        // order of the data, but handling this case is outside the scope
        // of this implementation
        if (sourceSet.byteOrder != destSet.byteOrder) return false;

        destSet.getOrCreateExifDirectory();

        // Go through the source directories
        List<?> sourceDirectories = sourceSet.getDirectories();
        for (int i=0; i<sourceDirectories.size(); i++)
        {
            TiffOutputDirectory sourceDirectory = (TiffOutputDirectory)sourceDirectories.get(i);
            TiffOutputDirectory destinationDirectory = getOrCreateExifDirectory(destSet, sourceDirectory);

            if (destinationDirectory == null) continue; // failed to create

            // Loop the fields
            List<?> sourceFields = sourceDirectory.getFields();
            for (int j=0; j<sourceFields.size(); j++)
            {
                // Get the source field
                TiffOutputField sourceField = (TiffOutputField) sourceFields.get(j);

                // Check exclusion list
                if (excludedFields != null && excludedFields.contains(sourceField.tagInfo))
                {
                    destinationDirectory.removeField(sourceField.tagInfo);
                    continue;
                }

                // Remove any existing field
                destinationDirectory.removeField(sourceField.tagInfo);

                // Add field
                destinationDirectory.add(sourceField);
            }
        }

        // Save data to destination
        tempStream = new BufferedOutputStream(new FileOutputStream(tempFile));
        new ExifRewriter().updateExifMetadataLossless(destFile, tempStream, destSet);
        tempStream.close();

        // Replace file
        if (destFile.delete())
        {
            tempFile.renameTo(destFile);
        }

        return true;
    }
    catch (ImageReadException exception)
    {
        exception.printStackTrace();
    }
    catch (ImageWriteException exception)
    {
        exception.printStackTrace();
    }
    catch (IOException exception)
    {
        exception.printStackTrace();
    }
    finally
    {
        if (tempStream != null)
        {
            try
            {
                tempStream.close();
            }
            catch (IOException e)
            {
            }
        }

        if (tempFile != null)
        {
            if (tempFile.exists()) tempFile.delete();
        }
    }

    return false;
}

private static TiffOutputSet getSanselanOutputSet(File jpegImageFile, int defaultByteOrder)
        throws IOException, ImageReadException, ImageWriteException
{
    TiffImageMetadata exif = null;
    TiffOutputSet outputSet = null;

    IImageMetadata metadata = Sanselan.getMetadata(jpegImageFile);
    JpegImageMetadata jpegMetadata = (JpegImageMetadata) metadata;
    if (jpegMetadata != null)
    {
        exif = jpegMetadata.getExif();

        if (exif != null)
        {
            outputSet = exif.getOutputSet();
        }
    }

    // If JPEG file contains no EXIF metadata, create an empty set
    // of EXIF metadata. Otherwise, use existing EXIF metadata to
    // keep all other existing tags
    if (outputSet == null)
        outputSet = new TiffOutputSet(exif==null?defaultByteOrder:exif.contents.header.byteOrder);

    return outputSet;
}

private static TiffOutputDirectory getOrCreateExifDirectory(TiffOutputSet outputSet, TiffOutputDirectory outputDirectory)
{
    TiffOutputDirectory result = outputSet.findDirectory(outputDirectory.type);
    if (result != null)
        return result;
    result = new TiffOutputDirectory(outputDirectory.type);
    try
    {
        outputSet.addDirectory(result);
    }
    catch (ImageWriteException e)
    {
        return null;
    }
    return result;
}

Note that we used the lossless Sanselan function (updateExifMetadataLossless), so Sanselan can copy all existing EXIF metadata intact, regardless of whether Sanselan “knows” about custom EXIF fields or marker notes that may appear in the data. You can use the lossy Sanselan function (updateExifMetadataLossy) if you need Sanselan to drop custom fields and marker notes.

Conclusion

Hopefully this solution is useful to you when faced with situations where you need to copy EXIF metadata.  Let us know about any questions or problems in the comments.

Handling Large Images on Android

Dec 7, 2012   //   by Theo   //   Blog  //  2 Comments

The Challenge

Did you ever encounter one of these errors while processing large images on Android?

OutofMemoryError: bitmap size exceeds VM budget

If you have done a fair amount of image work on Android, I am sure you did. This is one of the most common errors when working with images.

Why does it happen?

The Android Java API requires loading the entire image into memory for processing. Unfortunately sometimes there is not enough memory to load the entire image into memory, so the operation fails.

Whether or not there is sufficient memory depends on a variety of factors including:

  • The amount of heap memory allocated for the process. This differs from device to device and can be as little as 16MB on the original TMobile G1 and as much as 64 MB or more on more recent models. The amount of heap memory is set in the device’s firmware.
  • The amount of memory usage within the same process. If your app already loaded a lot of bitmaps, then there may not be sufficient memory to load additional bitmaps, and some old bitmaps may need to be removed from memory to do so.
  • The amount of fragmentation in the heap. The total number of bytes required to load your image may exist on the heap, but the heap memory may be fragmented, meaning that the free memory is split into several chunks. If no sufficiently large contiguous area can fit your image, then allocation will fail.

How to fix it

This article assumes that you can sacrifice a bit of image resolution in order to complete processing using the Java API, which is memory bound. If this is not acceptable, then a more complex solution such as using a C/C++ library and the NDK is probably appropriate.

This article also assumes that you are managing memory correctly. Improper memory management such as progressively leaking images when orientation changes or failing to unload adapter images that are not displayed on screen can also substantially reduce the available memory and lead to frequent OutOfMemoryError conditions.

The Solution

To get rid of the OutOfMemoryError errors we will reduce the size of the image that is loaded in the memory, sacrificing some image resolution in the process.

A natural question at this point is: what is the proper size that will guarantee that the image will fit into memory? Unfortunately there is no magic size that is guaranteed to work on every device — each device has varying amounts of memory.

Faced with this, it may be tempting to try to determine the amount of remaining free memory. Unfortunately this is not likely to be successful. The Java API makes it difficult to get a reliable number. Even if you could get a reliable number, there is no guarantee that the available free memory is contiguous (see heap fragmentation issue above).

A better approach is to rely on brute force. Repeatedly attempt to load the image into memory and progressively use a smaller image size if the load fails, until the load finally succeeds. A failed load will tend to fail fast, because allocating memory is one of the first steps in processing.

This algorithm is demonstrated below in the context of rotating an image. We increment inSampleSize with each iteration to reduce the image size, and continue as for as long as an OutOfMemoryError occurs. If your program is managing memory properly and you don’t have other large images loaded into memory, this approach requires no more than 3 iterations.

private boolean rotateBitmap(File inFile, File outFile, int angle)
		throws FileNotFoundException, IOException
{
    // Declare
    FileInputStream inStream = null;
    FileOutputStream outStream = null;

    // Create options
    BitmapFactory.Options options = new BitmapFactory.Options();

    // Create transform matrix
    Matrix matrix = new Matrix();
    matrix.postRotate(angle);

    // Increment inSampleSize progressively to reduce image resolution and size. If
    // the program is properly managing memory, and you don't have other large images
    // loaded in memory, this loop will generally not need to go through more than 3
    // iterations. To be safe though, we stop looping after a certain amount of tries
    // to avoid infinite loops
    for (options.inSampleSize = 1; options.inSampleSize <= 32; options.inSampleSize++)
    {
        try
        {
            // Load the bitmap from file
            inStream = new FileInputStream(inFile);
            Bitmap originalBitmap = BitmapFactory.decodeStream(inStream, null, options);

            // Rotate the bitmap
            Bitmap rotatedBitmap = Bitmap.createBitmap(originalBitmap, 0, 0, originalBitmap.getWidth(), originalBitmap.getHeight(), matrix, true);

            // Save the rotated bitmap
            outStream = new FileOutputStream(outFile);
            rotatedBitmap.compress(CompressFormat.JPEG, 100, outStream);
            outStream.close();

            // Recycle the bitmaps to immediately free memory
            originalBitmap.recycle();
            originalBitmap = null;
            rotatedBitmap.recycle();
            rotatedBitmap = null;

            // Return
            return true;
        }
        catch (OutOfMemoryError e)
        {
            // If an OutOfMemoryError occurred, we continue with for loop and next inSampleSize value
        }
        finally
        {
            // Clean-up if we failed on save
            if (outStream != null)
            {
                try
                {
                    outStream.close();
                }
                catch (IOException e)
                {
                }
            }
        }
    }

    // Failed
    return false;
}

This approach of trapping the OutOfMemoryError and progressively reducing image resolution can also work for other transforms using the Java API such as hue changes, greyscale conversions and more.

Conclusion

Hopefully this article is helpful in helping you process large images using the Java API. Let us know about any questions or comments in the section below.

Fixing the broken Honeycomb and ICS webview

May 15, 2012   //   by Theo   //   Blog  //  1 Comment

Background

HTML-based Android apps typically load their bundled HTML files into webviews via special android_asset URLs. On Android 3.0+ and Android 4.0, if these android_asset URLs contain parameters (e.g. file:///android_asset/mypage.html?test=test) or anchors (e.g. file:///android_asset/mypage.html#test), the webview incorrectly displays a page not found error — even though the file exists. (UPDATE: This issue is finally fixed in Android 4.1).

This is known as Android Issue 17535 and it seriously breaks the webview. For app developers, this means not being able to use parameters or anchors in HTML builds. Needless to say, this is a major impediment. As of right now (May 2012), there is no workaround.  Some suggest using HTML5 local storage to pass parameters, but that requires a lot of custom work and modifications to your HTML build just to support the Android platform.

If you are looking for a better way, then this article explains a workaround that fixes this issue completely and makes the webview work correctly with asset URLs that contain both parameters and anchors. The fix is provided as a JAR for easy inclusion into your projects.

How does it work?

This workaround extends the WebView and WebViewClient classes to create an assets cache in your app’s data directory and instruct the webview to load pages from this cache. This effectively side-steps the assets load issues, because the HTML files are now loaded from the cache.

With this workaround, whenever a new asset request is processed in the webview, the following things happen:

  • The asset URL is translated into a corresponding cache URL. For example, file:///android_asset/test.html becomes file:///data/data/com.yourcompany.yourpackage/webviewfix/test.html.
  • The asset file is copied from the assets directory to the cache directory.
  • The webview gets redirected to the new file in the cache.

How do I use this in my app?

To include this workaround in your app follow these steps:

  1. Download a copy of the JAR from the following URL: https://github.com/bricolsoftconsulting/WebViewIssue17535Fix/raw/master/bin/webviewissue17535fix.jar
  2. Add the JAR to your project. How you do this will depend on the specific tools you use. For example, in Eclipse you can right click on your project in Package Explorer, select Properties from the context menu, select Java Build Path on the left hand side of the Properties dialog, click the Add External JAR button in the dialog and finally browse to and select the WebViewIssue17535Fix.jar file.
  3. Locate all WebView references in your XML layouts and change them to com.bricolsoftconsulting.webview.WebViewEx.
  4. When initializing each WebViewEx object in your code, provide a reference to a WebViewClientEx object:
    mWebView = (WebViewEx) findViewById(R.id.webview);
    mWebView.setWebViewClient(new WebViewClientEx(WebViewActivity.this));
    

    If you want to override the shouldOverrideUrlLoading and shouldInterceptRequest events in WebViewClient, you must ensure that your implementation calls the superclass implementation and does not interfere with its functioning. Sample code is posted below:

    mWebView = (WebViewEx) findViewById(R.id.webview);
    
    mWebView.setWebViewClient(new WebViewClientEx(WebViewActivity.this)
    {
        @Override
        public boolean shouldOverrideUrlLoading(WebView view, String url)
        {
            // Special handling for shouldOverrideUrlLoading
            // Make sure that we call the base class implementation and do not interfere
            // with the base class redirects
            boolean redirected = super.shouldOverrideUrlLoading(view, url);
    
            // Do your own redirects here and set the return flag
            if (!redirected)
            {
                // Redirect HTTP and HTTPS urls to the external browser
                if (url != null && URLUtil.isHttpUrl(url) || URLUtil.isHttpsUrl(url))
                {
                    view.getContext().startActivity(new Intent(Intent.ACTION_VIEW, Uri.parse(url)));
                    redirected = true;
                }
            }
            return redirected;
        }
    
        @Override
        public WebResourceResponse shouldInterceptRequest(WebView view, String url)
        {
            // Special handling for shouldInterceptRequest
            // Make sure that we call the base class implementation
            WebResourceResponse wrr = super.shouldInterceptRequest(view, url);
    
            // Do your own resource replacements here
            if (wrr == null)
            {
                // wrr = new WebResourceResponse...
            }
            return wrr;
        }
    });
    
  5. Make sure that your Android manifest declares a target SDK version of at least 11:
    <uses-sdk android:targetSdkVersion="11" />
    

    If using Eclipse, also make sure that you change the Android target in the project properties.

  6. Add the appropriate webview permissions to your manifest, as shown below:
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
    <uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
    

    Optionally, if you want to set the cache path to the SDCARD:

    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
    

Conclusion

That’s it — once you add the fix to your project, this webview issue should be completely gone. Feel free to try our Github demo project and see for yourself.

How to enable native heap tracking in DDMS

Apr 25, 2012   //   by Theo   //   Blog  //  4 Comments


Background

When searching for memory leaks, we often need to examine how a process’ memory usage evolves over time. DDMS provides easy-to-use features to track Java virtual machine heap usage. However, when it comes to native heap usage, DDMS make things much harder.  This article provides a step-by-step guide to help you configure native heap tracking in DDMS.

Pre-requisites

In order to configure an Android device to track native heap usage, you will need:

  • a Mac, Linux or Windows machine with the Android SDK and ADB installed
  • an Android device with either:
    • an eng or userdebug Android build OR
    • a regular user build that has been rooted and has SuperUser installed.

Instructions on how to root specific phone models are available elsewhere on the web, and the rooting procedure usually includes the installation of SuperUser.

Device Setup

Libc Installation

All the native memory allocation functions (malloc, calloc, etc.) reside in Android’s libc library. To track heap allocations we need special versions of this library, which can log each allocation as it happens, and report back the data when required. Such special versions of libc (called libc_malloc_debug_leak.so and libc_malloc_debug_qemu.so) are included with Android, but only in the eng and userdebug Android builds. If you have either  an eng or userdebug build on your phone, then no additional steps are needed for libc installation and you can jump to the next section.

If you have a regular user build, you can add the missing libraries as follows:

  1. Download the version of CyanogenMod ROM that most closely matches your phone’s model and minor Android OS version (e.g 2.3, 4.0). If your exact model is not supported, try to find the closes match by processor type (e.g. Tegra2) and minor Android OS version. Once you have the right image, extract the libc_malloc_debug_leak.so and libc_malloc_debug_qemu.so from the system/lib directory.
  2. Copy the libc_malloc_debug_leak.so and libc_malloc_debug_qemu.so files via USB to the root of the SDCARD.
  3. Install BusyBox so you can later use the cp command to copy files.
  4. From your computer”s command prompt, log into the device shell and get a list of mounted devices using the mount command:
    adb shell
    su
    mount
    
  5. The mount output will list all mounted partitions with the corresponding devices and file system types. Examine the output and make a note of the the device and file system type for your system partition. As an example, the lines below show the system entries for two different Android phones:
    LG G2x:
    ...
    /dev/block/mmcblk0p1 /system ext3 ro,noatime,errors=continue,data=ordered 0 0
    ...
    
    Nexus S:
    ...
    /dev/block/platform/s3c-sdhci.0/by-name/system /system ext4 ro,relatime,barrier=1,data=ordered 0 0
    ...
    

    The first phone has /dev/block/mmcblk0p1 as the device and ext3 as the file system type. The second phone has /dev/block/platform/s3c-sdhci.0/by-name/system as the device and ext4 as the file system type.

  6. Using the partition device name and file system type you obtained in step 5 above, remount the system partition in read-write mode. The section below shows the remount command for the two phones above:
    LG G2x:
    mount -o remount,rw -t ext3 /dev/block/mmcblk0p1 /system
    
    Nexus S:
    mount -o remount,rw -t ext4 /dev/block/platform/s3c-sdhci.0/by-name/system /system
    
  7. Copy the two libc files from the SDCARD to the /system/lib directory:
    cp /sdcard/libc_malloc_debug_leak.so /system/lib/libc_malloc_debug_leak.so
    cp /sdcard/libc_malloc_debug_qemu.so /system/lib/libc_malloc_debug_qemu.so
    
  8. Set the proper permissions:
    chmod 0644 /system/lib/libc_malloc_debug_leak.so
    chmod 0644 /system/lib/libc_malloc_debug_qemu.so
    
  9. Reboot the phone to reset the system partition to read only.

Property Configuration

Once the device has the new libc files installed, you need to tell the system to use the new libc files for memory allocation.

  1. Set the libc.debug.malloc property to one of the supported values below, depending on your needs. If you are using a device with a rooted user build, then you must perform these commands from the device’s root prompt (use su to get root privileges, as shown below).
    adb shell
    su
    setprop libc.debug.malloc 1
    

    Supported property values:

    1  - perform leak detection
    5  - fill allocated memory to detect overruns
    10 - fill memory and add sentinels to detect overruns
    20 - use special instrumented malloc/free routines for the emulator
    
  2. Restart the application framework from the same root prompt. For this to work you must use a prompt with root privileges.
    stop
    start
    

    If the commands are successful, your device will seem to reboot 1-2 seconds after the commands were issued. However, it’s not a full reboot, and you will notice that the root prompt you opened earlier remains connected.

  3. From the same prompt used earlier, check that the property was successfully set:
    getprop
    

    The command above will display a list of custom properties, which should contain a line like the following:

    ...
    [libc.debug.malloc]: [1]
    ...
    
  4. As an additional check, issue the ls command from the device prompt. If the setup was successful, then you should see something like the following line in the device log:
    05-10 20:34:01.494: I/libc(6978): ls using MALLOC_DEBUG = 1 (leak checker)
    

Computer Setup

By default, DDMS does not display the native heap tab. To enable that tab, we must modify the DDMS configuration.

  1. Open the ddms.cfg file located at ~/.android/ddms.cfg.
  2. At the bottom of the file, add the following line:
    ...
    native=true
    
  3. Save and close the configuration file.
  4. Open the stand alone DDMS tool from the Android SDK tools folder. Make sure that no Eclipse instance is open, since only one instance of DDMS can operate at a time.
  5. Select one of the processes from the list of running processes, and in right hand side panel open the Native Heap tab. Press the Snapshot Current Native Heap Usage button and you should see a list of allocations similar to the one shown in the graphic at the top of this article.

Troubleshooting

The native tab is not populated with data.

Couple of things to check:

  • Use adb shell followed by ls /system/lib/libc* to confirm that  libc_malloc_debug_leak.so and libc_malloc_debug_qemu.so are present on the device.
  • Perform a computer restart. Also shut down the Android device and remove the battery for 20 seconds before restarting. Once that is complete, redo all the steps in the Property Configuration section. Double check that you are using the root command prompt  (adb shell followed by su) to issue all the shell commands, and that you are typing the commands correctly.
  • Make sure that after issuing the stop and start commands the device successfully resets the application framework (it should look like a reboot but without losing the adb connection).

When I issue the start command the adb connection breaks.

You are probably using the wrong versions of the libc files. Make sure that the libc files are compatible with your device’s processor and Android version. If you are using CyanogenMod to get the libc files, try to get them from the same phone model (or one with a similar processor) with the same Android OS level (2.3, 4.0, etc.)

The log shows the following error: “/system/bin/sh: Missing module /system/lib/libc_malloc_debug_leak.so required for malloc debug level 1”

You are missing the libc libraries required for debug. Please follow the steps in the Libc Installation section to correct the problem.

How to test onSaveInstanceState and onRestoreInstanceState on a real device

Dec 23, 2011   //   by Theo   //   Blog  //  5 Comments

Background

In its default configuration, the Android OS actively optimizes resources to achieve fast activity loading times. The system makes an effort to retain previously displayed activities in memory so they can be loaded quickly if they are needed again in the future. All previous activities are kept in their last known state.

Sometimes though, the system does not have enough memory and has to unload formerly used activities from memory. When it does this, it calls the onSaveInstanceState event, allowing the activity to save its current state (namely the member variables). Later, when the activity is needed again, the system brings it back into memory and calls the onRestoreInstanceState event to allow the activity to restore its state.

The Problem

The save instance state / restore instance state flow is a critical part of the activity lifecycle. Having errors in this area can lead to random app crashes for seemingly “unknown” reasons. As Android programmers, we need to test our apps to make sure that our activities correctly save and restore the state.

Though important, this testing is not easy — it’s difficult. By default, the Android OS manages memory internally, without any input from us. The OS single-handedly decides whether and when to unload your activities from memory and call the instance state events. We can’t schedule it.

(UPDATE: Starting with Ice Cream Sandwich Google has made manual testing easier by providing a debug option for this. You can enable Settings > Developer Options > Apps > Don’t Keep Activities to get the same behavior as described here. This helps for manual testing only… if you need to perform automatic testing keep reading.)

Or can we?

The Solution

SetAlwaysFinish is an Android app that causes the Android OS to predictably trigger the onSaveInstanceState and onRestoreInstanceState events. SetAlwaysFinish instructs the Android OS to unload from memory any activity that moves to the background, immediately triggering the onSaveInstanceState event. Given that the activity is always removed from memory, when the activity is needed again, the Android OS has to load it back into memory and fire the onRestoreInstanceState event.

This change means that the instance state events will always trigger, and they will trigger in a predictable manner. We can now test whether our activities correctly process the instance state events.

Usage

Here is how to use SetAlwaysFinish to test an app for instance state problems:

  1. Download the SetAlwaysFinish APK from our Github repository and install it on the device. For the install to work, you must make sure that the ‘Unknown Sources’ checkbox is checked under Settings > Applications.

    (UPDATE: In addition, if you have a device with JellyBean (Android 4.1) or above, the app will not work without being placed in the /system/app folder on the device. You will need to root your device, use adb shell to mount the system partition as read-write, copy the APK to /system/app, then set the system partition to read only again).
  2. Start SetAlwaysFinish on the test device and make sure that the Always Finish checkbox is checked.
  3. Run your app and go through every activity. Every time you create a new activity, the Android OS will unload the previous activity and trigger its onSaveInstanceState event. Every time you close an activity (rather than open a new activity on top of it), the Android OS will reload the activity immediately below the current activity in the activity stack and trigger its onRestoreInstanceState event.
  4. To test the activities at the topmost level in your app’s activity stack, you will need to have another external activity pop on top of them, so their onSaveInstanceState will trigger. The best way to do this is by launching a new app through ADB, as shown below.
    # Get shell access via ADB
    adb shell
    
    # Start the calculator application on the device
    am start -n com.android.calculator2/com.android.calculator2.Calculator
    
  5. Once you are done testing your app, you can return to the SetAlwaysFinish app and uncheck the Always Finish checkbox to allow Android to optimize for speed once again.

If you want to find out more about how the SetAlwaysFinish tool works, keep on reading. Otherwise, you have everything you need to test onSaveInstanceState and onRestoreInstanceState on a physical device.

How does it work?

The SetAlwaysFinish app uses hidden Android APIs. The exact setting is called (you guessed it!) AlwaysFinish and it is part of the hidden ActivityManagerNative class. To modify this setting, an app needs to have the ALWAYS_FINISH and WRITE_SETTINGS permissions.

(UPDATE: Starting with JellyBean, the ALWAYS_FINISH permission is granted to system apps only. An easy way for the SetAlwaysFinish app to become a system app is to place its APK in the /system/app directory as described in the usage section above, but keep in mind that this requires a rooted device.)

This setting applies system-wide for all apps running on the device. While you can use it for testing on your own devices, we do not recommend that you modify this setting on your customers’ devices.

The relevant code is located in the two functions below:

// Updates the system Always Finish setting
private void writeFinishOptions()
{
	try
	{
		// Due to restrictions related to hidden APIs, need to emulate the line below
		// using reflection:
		// ActivityManagerNative.getDefault().setAlwaysFinish(mAlwaysFinish);
		final Class   classActivityManagerNative = Class.forName("android.app.ActivityManagerNative");
		final Method     methodGetDefault = classActivityManagerNative.getMethod("getDefault");
		final Method     methodSetAlwaysFinish = classActivityManagerNative.getMethod("setAlwaysFinish", new Class[] {boolean.class});
		final Object     objectInstance = methodGetDefault.invoke(null);
		methodSetAlwaysFinish.invoke(objectInstance, new Object[]{mAlwaysFinish});
	}
	catch (Exception ex)
	{
		showAlert("Could not set always finish:\n\n" + ex, "Error");
	}
}

// Gets the latest AlwaysFinish value from the system and
// updates the checkbox
private void updateFinishOptions()
{
	mAlwaysFinish = Settings.System.getInt(getContentResolver(), Settings.System.ALWAYS_FINISH_ACTIVITIES, 0) != 0;
	mAlwaysFinishCB.setChecked(mAlwaysFinish);
}

Notice that we use:

android.app.ActivityManagerNative.getDefault().setAlwaysFinish(booleanValue);

to set the setting, and we use:

android.provider.Settings.System.getInt(getContentResolver(), Settings.System.ALWAYS_FINISH_ACTIVITIES, 0)

to read it back.

Conclusion

Feel free to grab a copy of the SetAlwaysFinish source code from our Github repository at:
https://github.com/bricolsoftconsulting/SetAlwaysFinish

If you need the complete APK, you can find it here:
https://github.com/bricolsoftconsulting/SetAlwaysFinish/raw/master/release/SetAlwaysFinish.apk

Credits

This utility draws inspiration (and code) from Google’s DevTools app found in the emulator. The DevTools app contains similar functionality — an ‘Immediately destroy activities’ checkbox under ‘Development Settings’. Unfortunately, due to its use of restricted permissions, the DevTools app does not work outside of the emulator.

Perfect Geocoding Zoom: Part 2

Nov 6, 2011   //   by Theo   //   Blog  //  1 Comment

The Problem

Ever used the built-in Geocoder class in Android to display a location on a map? If so, you may have found that the geocoder does not provide you with the viewport data needed to zoom the map appropriately. Given the reality of different size locations (countries, states, cities, buildings) how can we zoom the map appropriately to show the complete location on the screen, at a zoom level that’s not too high and not too low?

The Solution

GeocoderPlus is a geocoding library that uses the Google Maps Web Services APIs V3 to retrieve both location coordinates and viewport info. Thanks to the viewport info, the GeocoderPlus library can display locations on a map at the right zoom level.

In Part 1 of this post we looked at the inner workings of the library. In this post we will build an sample app that shows a map and allows the user to enter a location. When the user presses Go, the app calls the GeocoderPlus library to retrieve matching addresses. If there are multiple addresses that match, the app asks the user to select the desired address. Finally, the app displays the selected address at the correct zoom on the map.

Installation

GeocoderPlus is provided as a JAR library. Download the library from here and add it to your Android project like any other JAR library.

Interface

The GeocoderPlus interface is similar to that of the Android geocoder. The exported Geocoder class contains these methods:

public List <Address> getFromLocationName(String locationName)
	throws IllegalArgumentException, IOException

public List <Address> getFromLocationName (String locationName, int maxResults)
	throws IllegalArgumentException, IOException

Usage

The following method shows how to perform geocoding and handle any exceptions:

public List<Address> geocodeLocation(String locationName)
{
	// Geocode the location
	Geocoder geocoder = new Geocoder();

	try
	{
		List<Address> addresses = geocoder.getFromLocationName(locationName);
		return addresses;
	}
	catch (IllegalArgumentException e)
	{
		e.printStackTrace();
	}
	catch (IOException e)
	{
		e.printStackTrace();
	}

	return null;
}

The geocoding call is a blocking call, so proper Android programming dictates that it should not take place on the UI thread. The easiest way to transfer the call to a different thread is to use the AsyncTask class. The code shown below extends the AsyncTask class and creates a new class called GeocodeTask. Within the new class, onPreExecute uses showLocationPendingDialog to display an indeterminate progress dialog with a status message. The doInBackground function is executed on a separate thread and calls our geocodeLocation() function (shown earlier) to perform the geocoding work. Once geocoding is complete, onPostExecute uses cancelLocationPendingDialog to hide the indeterminate progress dialog, and then takes appropriate action based on the number of results. The user will see a warning if there are zero results, a location picker if there are multiple results and the actual location if there is one result.

public class GeocodeTask extends AsyncTask<String, Void, List<Address>>
{
	// Removed helper function implementations for brevity. You can see
	// the full code in the Github repository

	@Override
	protected void onPreExecute()
	{
		super.onPreExecute();
		showLocationPendingDialog();
	}

	@Override
	protected List<Address> doInBackground(String... args)
	{
		// Declare
		List<Address> addresses;

		// Extract parameters
		String locationName = args[0];

		// Geocode
		addresses = geocodeLocation(locationName);

		// Return
		return addresses;
	}
	@Override
	protected void onPostExecute(List<Address> addresses)
	{
		super.onPostExecute(addresses);
		cancelLocationPendingDialog();

		// Check the number of results
		if (addresses != null)
		{
			if (addresses.size() > 1)
			{
				// Determine which address to display
				showLocationPicker(addresses);
			}
			else
			{
				// Display address
				displayLocation(addresses.get(0));
			}
		}
		else
		{
			showAlert("No results found!", "Error");
		}
	}
};

To display the address on the map, we use the data in the Address class (latitude, longitude, viewport latitude span, viewport longitude span) to center the map and set the zoom.

private void displayLocation(Address address)
{
	// Get the map center
	GeoPoint mapCenter = new GeoPoint((int) (address.getLatitudeE6()), (int) (address.getLongitudeE6()));

	// Get the map controller
	MapController mapController = mMapView.getController();

	// Fix position
	mapController.animateTo(mapCenter);

	// Fix zoom
	mapController.zoomToSpan(address.getViewPort().getLatitudeSpanE6(), address.getViewPort().getLongitudeSpanE6());
}

Finally, we can invoke the GeocodeTask class and perform geocoding in the background as follows:

String locationName = "Rome";
GeocodeTask geocodeTask = new GeocodeTask();
geocodeTask.execute(locationName);

Conclusion

This post showed how to use the GeocoderPlus library to display an address at the appropriate zoom level on a map.

The source code for this post is on Github at:
http://www.github.com/bricolsoftconsulting/GeocoderPlusExample/

Perfect Geocoding Zoom: Part 1

Nov 4, 2011   //   by Theo   //   Blog  //  No Comments

The Problem

Ever used the built-in Geocoder class in Android to display a location on a map? If so, you may have found that the geocoder does not provide you with the viewport data needed to zoom the map appropriately. Given the reality of different size locations (countries, states, cities, buildings) how can we zoom the map appropriately to show the complete location on the screen, at a zoom level that’s not too high and not too low?

The Solution

In this post we will build a geocoding library called GeocoderPlus that uses the Google Maps Web Services APIs V3 to retrieve both location coordinates and viewport info. In the second part of this post, we will use the GeocoderPlus library to display locations on a map at the right zoom level. If you are simply interested in using the library without understanding the nuts and bolts, you can jump ahead to the second part.

The source code for this post is on Github at:
http://www.github.com/bricolsoftconsulting/GeocoderPlus/

GeocoderPlus

The GeocoderPlus library uses the Google Maps Web Services APIs V3 to retrieve both location coordinates and viewport info. The entire geocoding process consists of these steps:

  1. Create an appropriate request URL.
  2. Retrieve the data from the URL in JSON format.
  3. Parse the data from JSON into appropriate structures.

Step 1: Create the request URL

A Google Maps Web Services API V3 geocoding URL takes the form:

http://maps.googleapis.com/maps/api/geocode/output?parameters

The parameters used are:

Parameter Description
address The address you want to geocode.
sensor Indicates whether the device has a sensor (true / false).
language The language to use for the results.
region Specify a region to customize the results via region biasing. Region biasing gives priority to results in your region over results in other regions.

For example, to query the location for the city of Rome and customize the results for the English language with region biasing for Great Britain, the query is:

http://maps.googleapis.com/maps/api/geocode/json?address=Rome&sensor=true&language=en_gb®ion=gb

Armed with this knowledge, we can write a function that creates the URL and appends the required parameters to it.

// Constants
public static final String URL_MAPS_GEOCODE = "http://maps.googleapis.com/maps/api/geocode/json";
public static final String PARAM_SENSOR = "sensor";
public static final String PARAM_ADDRESS = "address";
public static final String PARAM_LANGUAGE = "language";
public static final String PARAM_REGION = "region";

// Members
Locale mLocale;
boolean mUseRegionBias = false;

private String getGeocodingUrl(String locationName)
{
	// Declare
	String url;
	Vector params;

	// Extract language from locale
	String language = mLocale.getLanguage();

	// Create params
	params = new Vector();
	params.add(new BasicNameValuePair(PARAM_SENSOR, "true"));
	params.add(new BasicNameValuePair(PARAM_ADDRESS, locationName));
	if (language != null && language.length() > 0)
	{
		params.add(new BasicNameValuePair(PARAM_LANGUAGE, language));
	}
	if (mUseRegionBias)
	{
		String region = mLocale.getCountry();
		params.add(new BasicNameValuePair(PARAM_REGION, region));
	}

	// Create URL
	String encodedParams = URLEncodedUtils.format(params, "UTF-8");
	url = URL_MAPS_GEOCODE + "?" + encodedParams;

	// Return
	return url;
}

Step 2: Retrieve JSON data from the URL

To retrieve the JSON data from the URL, we create a class called HttpRetriever. This class relies on two built-in classes: HttpClient and BasicResponseHandler. HttpClient issues GET requests and BasicResponseHandler processes the returned data and converts it to a string.

The section below shows the implementation of the HttpClient class. The main function we will use to retrieve data is retrieve().

package com.bricolsoftconsulting.geocoderplus.util.http;

import java.io.IOException;

import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.BasicResponseHandler;
import org.apache.http.impl.client.DefaultHttpClient;

public class HttpRetriever
{
	// Members
	DefaultHttpClient httpclient = null;

	// Constructor
	public HttpRetriever()
	{
		httpclient = new DefaultHttpClient();
	}

	// Document retrieval function
	public String retrieve(String url) throws ClientProtocolException, IOException
	{
		// Declare
		String response = null;

		// Connect to server and get JSON response
		HttpGet request = new HttpGet(url);
		ResponseHandler responseHandler = new BasicResponseHandler();
		response = httpclient.execute(request, responseHandler);

		// Return
		return response;
	}
}

Step 3: Parse the JSON data

At this point, the next hurdle is parsing the JSON content and storing it in Java objects. To accomplish this, we need to understand the format of the JSON data, define custom Java classes to hold the extracted data and create functions to parse the data. Let’s take a look at each point.

The JSON Format

The best way to understand the format of the JSON data is to look at the JSON string returned in response to our Rome query (see Step 1 above for URL).

{
   "results" : [
      {
         "address_components" : [
            {
               "long_name" : "Rome",
               "short_name" : "Rome",
               "types" : [ "locality", "political" ]
            },
            {
               "long_name" : "Rome",
               "short_name" : "RM",
               "types" : [ "administrative_area_level_2", "political" ]
            },
            {
               "long_name" : "Lazio",
               "short_name" : "Lazio",
               "types" : [ "administrative_area_level_1", "political" ]
            },
            {
               "long_name" : "Italy",
               "short_name" : "IT",
               "types" : [ "country", "political" ]
            }
         ],
         "formatted_address" : "Rome, Italy",
         "geometry" : {
            "bounds" : {
               "northeast" : {
                  "lat" : 42.17071890,
                  "lng" : 12.92975280
               },
               "southwest" : {
                  "lat" : 41.65587379999999,
                  "lng" : 12.23442660
               }
            },
            "location" : {
               "lat" : 41.89051980,
               "lng" : 12.49424860
            },
            "location_type" : "APPROXIMATE",
            "viewport" : {
               "northeast" : {
                  "lat" : 42.17071890,
                  "lng" : 12.92975280
               },
               "southwest" : {
                  "lat" : 41.65587379999999,
                  "lng" : 12.23442660
               }
            }
         },
         "types" : [ "locality", "political" ]
      }
   ],
   "status" : "OK"
}

As you can see above, the entire result is a JSON object. Inside the JSON object there is a status field and a results array. Each item in the array is an address. Each address is broken down into 2 sections: address components and geometry.  Geometry is further broken down into viewport and bounds (viewing areas) and location (address latitude and longitude).

New Java Classes

Now that we understand the JSON format, let’s define appropriate data structures for this data.

Address Class

Android’s built in Address class is based on the second version of the Google Maps Geocoding APIs. The V2 APIs have since been deprecated and replaced with V3 APIs that use different fields. Accordingly, we need to create our own Address class based on the V3 fields. We will use the following fields:

Field Description
Locale The locale specifying the language and region for the result.
Street Number The street number component of the address.
Premise The named location, usually a building or a collection of buildings with the same name.
SubPremise An entity below the premise level, can be a building or a collection of buildings with the same name.
Floor The floor component of the address, if any.
Room The room component of the address, if any.
Neighborhood The neighborhood for the location.
Locality The city or town.
SubLocality Political entity below city or town.
AdminArea First entity below country level.
SubAdminArea Second entity below country level.
SubAdminArea2 Third entity below country level.
CountryName The name of the country (e.g France, Belgium)
CountryCode The two letter country code (e.g. FR, BE)
PostalCode The postal code component of the address.
FormattedAddress The complete address formatted for display as a string.
ViewPort The viewport for the address, which indicates the area that we should use to view the location.
Bounds The viewport for the address, which indicates the area that we should use to view the location. For countries, the bounds include any territories and possessions, while viewport just contains the main area of the country.
Latitude The location’s latitude.
Longitude The location’s longitude.

The implementation of this class is straight-forward, and consists of creating members fields plus getters and setters for these fields. Please see the Address.java file on Github for details.

Area and Point Classes

In the JSON data, the viewport and bounds objects are identical data structures representing a geographical area. In Java, we will represent these structures using the Area and Point classes. The Area class designates a geographical area using the northeast and southwest corners. Each corner point in turn is represented using the Point class, which consists of latitude and longitude coordinates.

Once again, the implementation of these classes is straight-forward, and consists of member fields plus getters and setters. See the Github  source code for details.

JSON Parsing

GeocoderPlus uses a series of functions to parse the JSON tree. Processing starts at the top of the tree and moves down, progressively extracting data from the JSON tree and transferring it into Java objects.

  1. The top level function, getAddressesFromJSON(), operates on the entire JSON tree and looks for individual addresses.
  2. The second level function, getAddressFromJSON(), operates on each JSON address object and relies on lower levels to fill in an Address object.
  3. The third level functions, populateAddressComponentsFromJSON() and populateAddressGeometryFromJSON(), process the address components and geometry sections of each address. Notice how the structure of the parsing code mirrors the structure of the JSON.
  4. At the fourth level, the populateAddressComponentsFromJSON() function performs the field-by-field transfer of address components into the Address object. The populateAddressGeometryFromJSON() function transfers the viewport, bounds and location (latitude/ longitude) data using the getAreaFromJSON() function.

The entire parsing process is built in such a way that any missing field will not cause the entire process to halt. Many of the fields are in fact optional and depend on the type of address.

Conclusion

This post explained the inner workings of the GeocoderPlus library and how it uses the Google Maps Geocoding APIs V3 to provide geocoding results that contain viewport information. The second part of this post will explain how to use the library to display objects on a map at the just the right zoom level.

The source code for this post is on Github at:
http://www.github.com/bricolsoftconsulting/GeocoderPlus/

Pages:12»

Blog Categories