We have developed over 10 dApps so far, when it comes to interacting with smart contracts, we have a better experience working with the ethers library, mostly because it is the library that we use in our Smart Contracts project (we use hardhat there, which natively uses ethers). However both web3js and ethers do essentially the same, I'd have no problem working with either.
So far we haven't worked with Vue, and it looks like there are not many libraries to interact with the ethereum blockchain. We mostly use React / Next.js, and this is our library of preference: https://github.com/tmm/wagmi, it has hooks for everything you'll ever want.
Web3Modal would seem to be the way to go actually, you'd need to make a few tweaks to make it vue-ish but it can totally work.
We're also considering the option of forking your client package and make it framework agnostic so we can use it with React or Svelte in the future, but we're looking to make an MVP in the course of 6 months (yeah the project is big enough for the MVP to take 6 months), so we'll try to go on with Vue.