To determine which mathematical statement is not true, we will evaluate each option based on our understanding of factors and multiples.

1. The statement "The number of factors of a given number is finite" is true. For any positive integer, the number of divisors (factors) is limited. For instance, the number 12 has the factors 1, 2, 3, 4, 6, and 12, which are finite in number.
2. The statement "Every factor of a given number is less than or equal to the given number" is true. By definition, a factor of a number divides that number completely, which means it cannot be greater than the number itself. For example, factors of 10 are 1, 2, 5, and 10, all of which are less than or equal to 10.
3. The statement "Every factor of a given number is always greater than the given number" is false. As mentioned earlier, factors are numbers that divide a given number entirely, and such numbers cannot be greater than the number itself. This directly contradicts the correct understanding of factors.
4. The statement "Every multiple of a given number is greater than or equal to that number" is true. A multiple of a number is obtained by multiplying that number with an integer. For example, multiples of 5 are 5, 10, 15, etc., all of which are equal to or greater than 5.

Hence, the statement "Every factor of a given number is always greater than the given number" is incorrect. This is because factors should be less than or equal to the number itself, not greater.