Yeah 10 bit encodes are a huge PITA. I just switched from Boxee Box to an HTPC with XBMC and even on that powerful PC XBMC by itself couldn't play these files without macroblocking and other image artifacts. I finally had to add MPC-HC with MadVR as an external player and set XBMC to switch to it for any file with 10 bit or hi10p in the file name. The files play beautifully now and for anime I can definitely see the benefit. It just sucks that they decided to jump on this when there are very few options to play the files without issue currently.
Boxee Box firmware: Latest
PC to Boxee Box connection: SMB over gigabit wired Ethernet.
Boxee Box buffers on BR MKVs and ISOs.
Streaming BR MKVs and ISO rips with full menus to Dune with no issues.
Streaming BR MKVs and ISO rips to two HTPCs using XBMC with no issues
PC Storing Media: Vista Home 64 bit, dual core (2.5GHz), 4GB RAM
Storage for media: 4 external WD Elements drives. 4 Hitachi 3TB drives
Router: NetGear WNR3500L-100NAS Wireless-N Gigabit Router