There is a calculation, but you need a capacitance meter, or you can find it empirically by measuring the rise time on your bus, but you need a scope for that. Generally, if you need to lower resistance for it to work, your capacitance has risen more than expected, slowing the rise time. Decreasing R gets it back up. Consider using wire with lower capacitance. If you are just experimenting, and not too concerned about EMI or power consumption, I would find whatever value works, then reduce it another 20% for margin. It's normally quite forgiving, you would probably need to be below 1.5K @ 5V before going too far.
I2C Bus Pullup Resistor Calculation from TI