ffpmeg is the only video codec XBMC/Boxee uses and cannot be changed
1080p mkv playback is choppy unless CoreAVC codec is used which it cannot be for XBMC.
720p files all ready play well but 1080p files often do not play smoothly for many users without a high-end setup.
This shows how to change one line of code in ffmpeg allowing hassle-free 1080p playback
Solution gleamed after I changed the setting in VLC:
* Open Tools > preferences
* Tick “All” under “Show settings in the lower right corner
* Expand "Input/Codec"
* Go to "other codecs" subcategory
* Select "FFmpeg"
* Select “skip the loop filter for H.264 decoding” under "All"
* Restart VLC
Adapted to Boxee (Windows 7/Vista):
*Extract the attached file "advancedsettings.zip" to C:\Users\<username>\AppData\Roaming\BOXEE\userdata \advancedsettings.xml
This file may not exist so adding it is fine. Also if you all ready have an advancedsettings.xml just open both and choose to merge the lines to your existing configuration!
Folder location in XBMC is C:\Users\<username>\AppData\Roaming\XBMC\userdata\ advancedsettings.xml
Please say thanks if this helps or let me know how we can make this more informative
***Huge thanks to cptspliff from the Official XBMC IRC channel for finding this setting and determining its needed value