Using Google Vision AI for Images scanning in Java/Groovy
Google's Vision AI is a great ML service which has pre-trained Vision models that helps you detecting labels from your Images. Google provided good examples in different programming languages to get started using their Vision AI service.
I have started using in Java/Groovy environment for getting labels from the images. I have started using their sample code from here https://github.com/GoogleCloudPlatform/java-docs-samples/blob/master/vision/cloud-client/src/main/java/com/example/vision/QuickstartSample.java
Firstly we need to add the Java dependency libraries, they are available in Maven Central and if you can add by following way for Gradle
compile 'com.google.cloud:google-cloud-vision:1.77.0'
Once i got the library i started using the code from their GitHub and ran into the following issue
Exception in thread "main" 21:20:14.423 [grpc-nio-worker-ELG-1-3] DEBUG io.grpc.netty.shaded.io.grpc.netty.NettyClientHandler - [id: 0x4c3546d3, L:/192.168.0.102:59965 - R:vision.googleapis.com/172.217.163.42:443] OUTBOUND GO_AWAY: lastStreamId=0 errorCode=0 length=0 bytes=
com.google.api.gax.rpc.PermissionDeniedException: io.grpc.StatusRuntimeException: PERMISSION_DENIED: Cloud Vision API has not been used in project 563584335869 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/vision.googleapis.com/overview?project=563584335869 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:55)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60)
at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:68)
at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1070)
at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1139)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:958)
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:748)
at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:515)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:490)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:700)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:399)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:500)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:65)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:592)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$700(ClientCallImpl.java:508)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:632)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Suppressed: com.google.api.gax.rpc.AsyncTaskException: Asynchronous task failed
at com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:57)
at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112)
at com.google.cloud.vision.v1.ImageAnnotatorClient.batchAnnotateImages(ImageAnnotatorClient.java:210)
at com.google.cloud.vision.v1.ImageAnnotatorClient.batchAnnotateImages(ImageAnnotatorClient.java:187)
at com.foo.service.VisionAPIService.main(VisionAPIService.java:47)
Caused by: io.grpc.StatusRuntimeException: PERMISSION_DENIED: Cloud Vision API has not been used in project 563584335869 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/vision.googleapis.com/overview?project=563584335869 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
at io.grpc.Status.asRuntimeException(Status.java:533)
... 23 more
From the error i know whats going wrong, i missed adding my API Keys, unfortunately Google examples didn't provided good information about that. After searching their documentation a bit i came to know one way of adding the APIs is by providing the environment variable GOOGLE_APPLICATION_CREDENTIALS with path to the JSON file that contains service account key.
But unfortunately that is something i can't do in my case as i its very hard to change my build steps and add this as environmental variable to my application. So i have created my own utility where i am adding this to environment using Java code itself that too with in the Vision AI wrapper. Following is the final wrapper code which takes an image path and return List of Labels.
package com.foo.service;
import java.lang.reflect.Field;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import javax.annotation.PostConstruct;
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import com.google.cloud.vision.v1.AnnotateImageRequest;
import com.google.cloud.vision.v1.AnnotateImageResponse;
import com.google.cloud.vision.v1.BatchAnnotateImagesResponse;
import com.google.cloud.vision.v1.EntityAnnotation;
import com.google.cloud.vision.v1.Feature;
import com.google.cloud.vision.v1.Feature.Type;
import groovy.util.logging.Slf4j
import com.google.cloud.vision.v1.Image;
import com.google.cloud.vision.v1.ImageAnnotatorClient;
import com.google.cloud.vision.v1.ImageSource;
@Service
@Slf4j
public class VisionAPIService {
private static final String JAVA_LANG_PROCESS_ENVIRONMENT = "java.lang.ProcessEnvironment"
private static final String THE_ENVIRONMENT = "theEnvironment"
private static final String THE_CASE_INSENSITIVE_ENVIRONMENT = "theCaseInsensitiveEnvironment"
private static final String JAVA_UTIL_COLLECTIONS$_UNMODIFIABLE_MAP = 'java.util.Collections$UnmodifiableMap'
private static final String M = "m"
@Value('${google.application.credentials}')
String credentialsPath
ImageAnnotatorClient vision
@PostConstruct
void init(){
HashMap<String, String> env = new HashMap<>()
env.put("GOOGLE_APPLICATION_CREDENTIALS", credentialsPath)
try {
setEnv(env)
vision = ImageAnnotatorClient.create()
} catch (Exception e) {
log.error('exception while initializing google vision api', e)
}
}
public List<String> getLabels(String imagePath) {
log.info('Running Vision API on image: {}', imagePath)
List<String> result = []
List<AnnotateImageRequest> requests = new ArrayList<>();
ImageSource imageSource = ImageSource.newBuilder().setImageUri(imagePath).build();
Image img = Image.newBuilder().setSource(imageSource).build();
Feature feat = Feature.newBuilder().setType(Type.LABEL_DETECTION).build();
AnnotateImageRequest request = AnnotateImageRequest.newBuilder().addFeatures(feat).setImage(img).build();
requests.add(request);
BatchAnnotateImagesResponse response = vision.batchAnnotateImages(requests);
List<AnnotateImageResponse> responses = response.getResponsesList();
for (AnnotateImageResponse res : responses) {
if (res.hasError()) {
log.error("Error: {}", res.getError().getMessage());
}else{
for (EntityAnnotation annotation : res.getLabelAnnotationsList()) {
annotation.getAllFields().each{ k,v ->
log.debug("%s : %s\n", k, v.toString());
if(k.getName() == 'description'){
String label = v.toString().toLowerCase();
label.split(' ').each{nv -> result.add(nv)}
}
}
}
}
}
return result;
}
protected static void setEnv(Map<String, String> envData) throws Exception {
try {
Class<?> processEnvironmentClass = Class.forName(JAVA_LANG_PROCESS_ENVIRONMENT);
Field theEnvironmentField = processEnvironmentClass.getDeclaredField(THE_ENVIRONMENT);
theEnvironmentField.setAccessible(true);
Map<String, String> env = (Map<String, String>) theEnvironmentField.get(null);
env.putAll(envData);
Field theCaseInsensitiveEnvironmentField = processEnvironmentClass.getDeclaredField(THE_CASE_INSENSITIVE_ENVIRONMENT);
theCaseInsensitiveEnvironmentField.setAccessible(true);
Map<String, String> cienv = (Map<String, String>) theCaseInsensitiveEnvironmentField.get(null);
cienv.putAll(envData);
} catch (NoSuchFieldException e) {
Class[] classes = Collections.class.getDeclaredClasses();
Map<String, String> env = System.getenv();
for(Class cl : classes) {
if(JAVA_UTIL_COLLECTIONS$_UNMODIFIABLE_MAP.equals(cl.getName())) {
Field field = cl.getDeclaredField(M);
field.setAccessible(true);
Object obj = field.get(env);
Map<String, String> map = (Map<String, String>) obj;
map.clear();
map.putAll(envData);
}
}
}
}
}