iPad HTTP Debugging with Charles

After noticing that the caching in iPad Safari seemed a little funky, I made an effort to decipher some of the logic used by the browser cache. I didn’t get very far, but in the process I figured out how to route my iPad HTTP traffic through a web debugger on my laptop. It turns out it was very easy to do (although I’m sure there is a more complicated way to go about it).

What follows is a simple step-by-step for connecting your iPad to an HTTP debugging proxy. The main requirement is that your desktop/laptop and iPad be on the same wireless network. Then it’s just a matter of telling your iPad to use your desktop as an HTTP proxy.

I’m going to walk through how to do this on a Mac with the highly recommended HTTP proxy app Charles, but the same basic steps should also work for a Windows HTTP debugger such as Fiddler. (Update: Fiddler users, see Eric Lawrence’s comment below.) Additionally, the steps are the same for connecting your iPhone or iPod Touch.

1. Get your system IP address for your desktop

The easiest way to do this on a Mac is to Spotlight your way to the “Network Utility” app. Make sure you’re on your Wi-Fi network and take note of the IP address. This is your desktop’s local IP.

iPad HTTP Proxy Network Utility

2. Tell your iPad about your desktop IP

On your iPad, go to Settings > Wi-Fi and click the arrow next your Wi-Fi connection. Under “HTTP Proxy”, select the Manual button and enter your desktop’s IP address in the Server field. Then enter “8888″ in the Port field. Close out.

iPad HTTP proxy settings

3. Configure Charles

Open Charles and select Proxy > Proxy Settings. Under the Proxies tab, make sure the HTTP proxy value is set to 8888. Then click the “Mac OS X” tab and check “Enable Mac OS X Proxy” and “Use HTTP Proxy”. Click OK.

Charles proxy settings

4. Start capturing HTTP traffic

Click the Record button in Charles and then load a web page in iPad Safari. You should see the following alert asking you if you want to allow your iPad to connect to Charles. Select “Allow”.

charles proxy alert

Now all of your HTTP traffic in iPad Safari should be recorded by Charles.

When you’re finished doing all of your fun HTTP debugging, don’t forget to turn off the HTTP proxy in the Wi-Fi settings of your iPad.

Update (04/08/13): Having trouble getting this to work? Are you behind a corporate firewall? Read this workaround.

Both comments and pings are currently closed.

Discussion

This is incredibly valuable. I haven’t tried it, but am tucking this away for the next time I want to observe HTTP traffic on my mobile device. Remember that proxies can change browser behavior (# of connections, maybe even gzip), so keep that in mind.

Excellent work! If this ends up affecting behavior too much, what I’ve done in the past is turn on routing on a Linux/FreeBSD box, set the iPad default gateway to the box’s IP and use tcpdump to capture the traffic so it can be viewed in wireshark.

Charles is a great tool but as Steve mentions it can influence behaviour – we use it regularly and notice it can influence performance at the client, we had some really strange flash behaviour with it (not that it will happen with the iPad of course)

I’d like to add that charles debugging proxy is excellent in web development as well. That goes for ajax as well as to analyze traffic/headers and general http behavior.

All,
Thanks very much for the comments and the warnings about the proxy potentially interfering with the browser’s behavior.

Tom Rankin

This is a great technique, I’ve been using it for a while to debug iPhone traffic using Fiddler2 on the PC. This is also a great way to reverse engineer traffic from other iPhone apps that communicate back to their servers over http because that traffic will end up going over the proxy as well.

Thanks for the Fiddler mention! The only configuration you need to do for Fiddler is tick the “Allow Remote Computers to Connect” inside Tools > Fiddler Options > Connections. Restart Fiddler and you’re good to go!

@Eric
Thanks for the info (and thanks for Fiddler)! I updated the post with a link to your comment.

I did some experiments in the same way on my Mac with TcpCatcher (default proxy port is 8200). The tool is free and allows packets modification on the fly ;-)

Does any app on the iPad communicate on a port other than 80/443? If so, how would you capture /analyze that traffic?

Thanks for the excellent article.

Will,

I don’t know how to proxy non-HTTP traffic. I’m sure there’s a way though…

When I update the settings on my iPad as instructed, I get an error message when I try to access Safari that says “There was a problem communicating with the web proxy server (HTTP).”. What am I doing wrong?

Vikram

Great solution Rob, thanks for the same

Is there any way to use the throttling feature when the iPad is in this configuration? I have to run tests on behavior in different bandwidths/bitrates. Please share your thoughts.

is possible to do with pc/iPad

Raj,
Yes, see Eric Lawrence’s comment above how you can do it on a PC with Fiddler.

Rob,

I tried setting up my ipad with the above mentioned settings and am some how not able to see any traffic reaching my win xp machine. I ran wireshark on the adapter and dont see any packets hitting the adapter :( from the ipad.
Any thoughts would be appreciated.

Thanks
Anoop

Anoop,
Is your iPad running iOS 5? I tried this a few days ago after updating to iOS 5 and I had trouble getting it to work. I haven’t had a chance yet to research what the issue is but when I do I’ll report back.

Great tutorial, very useful. Worked great, I can finally monitor the odd video/audio loading behaviors of iOS.

Thanks!

James Gaffar

Has anyone had trouble getting this to work on a PC running Charles? It works like a charm on a Mac but I am having some trouble on a PC. Any help would be greatly appreciated.

Thanks!

@James,
Sorry, I’ve never used Charles on a PC (when I was on a PC I used Fiddler) and I’m not that familiar with Windows network config. Maybe try asking on Stack Overflow?

Rob! thanks for the helpful information haha. Lets get drinks sometime…

Hey Matt! That’s hilarious… the magical power of the internet. Drinks would be great. I’ll get in touch over gmail.

This approach doesn’t work on new iPad at all, always saying “There was a problem communicating with the web proxy server (HTTP).”
Anyone managed to figure it out?

mintyquinty

¡Muchas gracias! this solves a development issue that I’ve been going around in circles on for a while now. We have iPad 3′s on order, so we will see if @Nik’s problems arise for use then, but for now, a big thanks!

Thanks insanely useful!

good writeup. useful still :)

I would have liked to see if ti’s possible to *only* record the traffic from the iPad, but that’s about the only addition to the writeup I’d suggest.

Mohammed Ashfaq

Environment:
iPad 3,iOS 5.1

Followed the instructions mentioned by Rob and tried accessing the URL using safari browser. Displayed below error message “There was a problem communicating web proxy server (Http)”.

Any thoughts on this.

I refer to this page often. Currently running iOS 6.1.2 and it works like a charm. From home.

I have found that when I am at work, inside our corporate firewall, that the Manual HTTP Proxy won’t stick. All I have to do is exit that Settings panel and as soon as I come back to that panel the HTTP Proxy is set to ‘Off’. I have contacted our extensive IT dept and they claim that there is no blocking so I’m a tad baffled.

I have a workaround that is working for me and that is to create an ad hoc network on the Mac: http://www.dummies.com/how-to/content/how-to-set-up-an-ad-hoc-wireless-network.html

I am posting this just in case someone else has found this page useful, but has run into a similar problem. It has worked every time for me.

@Steve,
I’ve also had trouble getting this to work on my company’s network. But in my case the problem was different—the proxy setting stayed “on” but the traffic couldn’t proxy through correctly. I think I ended up using the same ad hoc network workaround that you mentioned. Anyway, thanks for the tip! I’ll update the post with a link to your comment.

Hi is there a way to configure charles to xbox as i am having difficulties, ive tried bridging the network but that doesnt work any suggestions would really help me Thanks

@Steve.

I have seen that behaviour when the proxy/wireless settings are applied by a Mobile Config file and someone resets the network settings.

Remove the Mobile Config file, Re-Apply the Mobile Config file and you should find that the wireless profile will now keep your proxy settings again.

I succeffully set up as instruct. Why i can’t see any HLS traffic (download of ts files) when watching video in Charles? Other http traffic have no problem of showing up in Charles.

Repost to be clear:

I succeffully set up as instruct. Thru Charles, why i can’t see any HLS traffic (download of ts files) when watching video? Other http traffic have no problem of showing up in Charles.

thx,

Thanks for the post, it works great !

Any idea on how to do this with a command line ?
I’m sure that pf ( http://www.openbsd.org/faq/pf/ ) or at least mitmproxy ( http://mitmproxy.org/doc/transparent/osx.html ) could do this for free, but I couldn’t find how :-(