It is a common misunderstanding that what someone once jokingly called the “Big Bang” theory describes the universe as expanding from a single point.
What General Relativity actually suggests is just that in the distant past the universe was very dense (but still of infinite extent) and very hot, with everything flying apart so that since then it has been becoming less dense but not actually “expanding” in the sense of having a boundary that is moving outwards.
With regard to the second question about whether an object can “remain stationary”, in the absence of gravity (ie in Special Relativity) there is no way for an observer to identify whether or not it is moving in any absolute sense. But it is possible to identify whether an object is moving relative to any particular other object, or to the centre of mass of all the other material in its visible universe (so long as that visible universe contains just a finite total mass). And in General Relativity this can be done by checking for isotropy (sameness in all directions) of the microwave background radiation coming from the “Big Bang”.