Gunma isn't exactly a hotbed of startup activity, but there are some web developers living there. Gunma.web is a community for web developers living in Gunma, so that would be a good place to start. You could try contacting the organisers and asking them for advice.
Another thing I'd try doing is getting involved with Rails Girls Tokyo. As a bilingual Rails developer you could be quite valuable, as I think there are international women who would like to participate in it. Getting involved with the developer community through events like this is your best bet to find a job.
It might also be possible for you to work remotely for a startup (see this list). Particularly if you are able to come in to Tokyo on a regular basis (say once a week), some companies can be flexible.
It is theoretically possible for your company to pay for your commute, as by Japanese law they can cover up to ¥100,000 per month before it is considered income. Typically they companies will cover something like up to ¥30,000 per month, but even if it isn't part of their standard compensation package, you could probably negotiate a lower salary in exchange for them covering your commute. That way, at least you wouldn't be paying income tax.
However, unless you're living within walking distance of Takasaki station, and working near Tokyo station, I think the commute would be pretty rough. Even in that optimal case, you'd be spending two hours a day on the Shinkansen.