AVhF"adZddlZddlmZddlmZddlmZddlm Z ddl m Z ddlmZdd l mZdd l mZdd l mZdd lmZdd lmZddlmZddlmZm Z m!Z!ddl"m#Z#edddddddddddddddd d!d"d#d$Z$dDd%e#e!e$fd&e#e!e$fd'e#e!e$fd(e#e!e$fd)e#e!e$fd*e#e!e$fd+e#e!e$fd,e#e!e$fd-e#e!e$fd.e%d/e#e!e$ffd0Z&ed1ejNe&Z(d%e#e!e$fd&e#e!e$fd'e#e!e$fd(e#e!e$fd)e#e!e$fd*e#e!e$fd+e#e!e$fd,e#e!e$fd-e#e!e$fd.e%d/e#e!e$ffd2Z)ed3ddddddddddddddd d!d"d#d$Z*dDd%e#e!e*fd4e#e!e*fd5e#e!e*fd)e#e!e*fd6e#e!e*fd,e#e!e*fd-e#e!e*fd.e%d/e#e!e*ffd7Z+ed8ejNe+Z,d%e#e!e*fd4e#e!e*fd5e#e!e*fd)e#e!e*fd6e#e!e*fd,e#e!e*fd-e#e!e*fd.e%d/e#e!e*ffd9Z-ed:ddddddddddddddd d!d"d#d$Z.dEd%e#e!e.fd4e#e!e.fd)e#e!e.fd-e#e!e.fd.e%d;e%d/e#e!e.ffd<Z/ed=ejNe/Z0d%e#e!e.fd4e#e!e.fd)e#e!e.fd-e#e!e.fd.e%d;e%d/e#e!e.ffd>Z1ed?ddddddddddddddd d!d"d#d$Z2dDd%e#e!e2fd@e#e!e2fdAe#e!e2fd-e#e!e2fd)e#e!e2fdBe#e!e2fdCe#e!e2fdDe#e!ejffd.e%d/e#e!e2ffdEZ4edFejNe4Z5d%e#e!e2fd@e#e!e2fdAe#e!e2fd-e#e!e2fd)e#e!e2fdBe#e!e2fdCe#e!e2fdDe#e!ejffd.e%d/e#e!e2ffdGZ6edHddddddddddddddd d!d"d#d$Z7dEd%e#e!e7fd4e#e!e7fd)e#e!e7fd,e#e!e7fd-e#e!e7fd.e%d;e%d/e#e!e7ffdIZ8edJejNe8Z9d%e#e!e7fd4e#e!e7fd)e#e!e7fd,e#e!e7fd-e#e!e7fd.e%d;e%d/e#e!e7ffdKZ:edLddddddddddddddd d!d"d#d$Z;dFd%e#e!e;fd&e#e!e;fd'e#e!e;fd(e#e!e;fdMe#e!e;fd)e#e!e;fd*e#e!e;fd+e#e!e;fd,e#e!e;fd-e#e!e;fd.e%dNe%d/e#e!e;ffdOZ<edPejNe<Z=d%e#e!e;fd&e#e!e;fd'e#e!e;fd(e#e!e;fdMe#e!e;fd)e#e!e;fd*e#e!e;fd+e#e!e;fd,e#e!e;fd-e#e!e;fd.e%dNe%d/e#e!e;ffdQZ>edRddddddddddddddd d!d"d#d$Z?dDd%e#e!e?fd&e#e!e?fd)e#e!e?fdSe#e!e?fdTe#e!e?fdUe#e!e?fd-e#e!e?fd.e%d/e#e!e?ffdVZ@edWejNe@ZAd%e#e!e?fd&e#e!e?fd)e#e!e?fdSe#e!e?fdTe#e!e?fdUe#e!e?fd-e#e!e?fd.e%d/e#e!e?ffdXZBedYddddddddddddddd d!d"d#d$ZCdDd%e#e!eCfdZe#e!eCfd[e#e!eCfd\e#e!eCfd)e#e!eCfd6e#e!eCfd]e#e!eCfd,e#e!eCfd-e#e!eCfd.e%d/e#e!eCffd^ZDed_ejNeDZEd%e#e!eCfdZe#e!eCfd[e#e!eCfd\e#e!eCfd)e#e!eCfd6e#e!eCfd]e#e!eCfd,e#e!eCfd-e#e!eCfd.e%d/e#e!eCffd`ZFedaddddddddddddddd d!d"d#d$ZGdFd%e#e!eGfd4e#e!eGfdbe#e!eGfd-e#e!eGfd)e#e!eGfdBe#e!eGfdCe#e!eGfdce#e!eGfd.e%dde%d/e#e!eGffdeZHedfejNeHZId%e#e!eGfd4e#e!eGfdbe#e!eGfd-e#e!eGfd)e#e!eGfdBe#e!eGfdCe#e!eGfdce#e!eGfd.e%dde%d/e#e!eGffdgZJedhddddddddddddddd d!d"d#d$ZKdFd%e#e!eKfd4e#e!eKfdbe#e!eKfd-e#e!eKfd)e#e!eKfdBe#e!eKfdCe#e!eKfdie#e!eKfdce#e!eKfd.e%dde%d/e#e!eKffdjZLedkejNeLZMd%e#e!eKfd4e#e!eKfdbe#e!eKfd-e#e!eKfd)e#e!eKfdBe#e!eKfdCe#e!eKfdie#e!eKfdce#e!eKfd.e%dde%d/e#e!eKffdlZNedmddddddddddddddd d!d"d#d$ZOdDd%e#e!eOfdSe#e!eOfdne#e!eOfd.e%d/e#e!eOff doZPedpejNePZQd%e#e!eOfdSe#e!eOfdne#e!eOfd.e%d/e#e!eOff dqZRedrddddddddddddddd d!d"d#d$ZSdFd%e#e!eSfd4e#e!eSfd)e#e!eSfd-e#e!eSfd]e#e!eSfd.e%dNe%d/e#e!eSffdsZTedtejNeTZUd%e#e!eSfd4e#e!eSfd)e#e!eSfd-e#e!eSfd]e#e!eSfd.e%dNe%d/e#e!eSffduZVedvddddddddddddddd d!d"d#d$ZWdDd%e#e!eWfd&e#e!eWfd)e#e!eWfdwe#e!eWfdTe#e!eWfdUe#e!eWfd-e#e!eWfd.e%d/e#e!eWffdxZXedyejNeXZYd%e#e!eWfd&e#e!eWfd)e#e!eWfdwe#e!eWfdTe#e!eWfdUe#e!eWfd-e#e!eWfd.e%d/e#e!eWffdzZZed{ddddddddddddddd d!d"d#d$Z[dDd%e#e!e[fd4e#e!e[fd)e#e!e[fdBe#e!e[fdCe#e!e[fd-e#e!e[fd.e%d/e#e!e[ffd|Z\ed}ejNe\Z]d%e#e!e[fd4e#e!e[fd)e#e!e[fdBe#e!e[fdCe#e!e[fd-e#e!e[fd.e%d/e#e!e[ffd~Z^edddddddddddddddd d!d"d#d$Z_dDd%e#e!e_fdSe#e!e_fdBe#e!e_fdCe#e!e_fdne#e!e_fd.e%d/e#e!e_ffdZ`edejNe`Zad%e#e!e_fdSe#e!e_fdBe#e!e_fdCe#e!e_fdne#e!e_fd.e%d/e#e!e_ffdZbedddddddddddddddd d!d"d#d$ZcdDd%e#e!ecfd[e#e!ecfd\e#e!ecfd)e#e!ecfd6e#e!ecfd]e#e!ecfd,e#e!ecfd-e#e!ecfd.e%d/e#e!ecffdZdedejNedZed%e#e!ecfd[e#e!ecfd\e#e!ecfd)e#e!ecfd6e#e!ecfd]e#e!ecfd,e#e!ecfd-e#e!ecfd.e%d/e#e!ecffdZfedddddddddddddddd d!d"d#d$ZgdDd%e#e!ejfd&e#e!ejfd'e#e!ejfd(e#e!egfd)e#e!egfd*e#e!egfd+e#e!egfd,e#e!egfd-e#e!egfd.e%fdZiedejNeiZjd%e#e!ejfd&e#e!ejfd'e#e!ejfd(e#e!egfd)e#e!egfd*e#e!egfd+e#e!egfd,e#e!egfd-e#e!egfd.e%fdZkedddddddddddddddd d!d"d#d$ZldDd%e#e!ejfd4e#e!ejfd5e#e!ejfd)e#e!elfd6e#e!elfd,e#e!elfd-e#e!elfd.e%fdZmedejNemZnd%e#e!ejfd4e#e!ejfd5e#e!ejfd)e#e!elfd6e#e!elfd,e#e!elfd-e#e!elfd.e%fdZoedddddddddddddddd d!d"d#d$ZpdEd%e#e!ejfd4e#e!ejfd)e#e!epfd-e#e!epfd.e%d;e%f dZqedejNeqZrd%e#e!ejfd4e#e!ejfd)e#e!epfd-e#e!epfd.e%d;e%f dZsedddddddddddddddd d!d"d#d$ZtdDd%e#e!ejfd@e#e!ejfdAe#e!ejfd-e#e!etfd)e#e!etfdBe#e!etfdCe#e!etfdDe#e!ejffd.e%fdZuedejNeuZvd%e#e!ejfd@e#e!ejfdAe#e!ejfd-e#e!etfd)e#e!etfdBe#e!etfdCe#e!etfdDe#e!ejffd.e%fdZwedddddddddddddddd d!d"d#d$ZxdEd%e#e!ejfd4e#e!ejfd)e#e!exfd,e#e!exfd-e#e!exfd.e%d;e%fdZyedejNeyZzd%e#e!ejfd4e#e!ejfd)e#e!exfd,e#e!exfd-e#e!exfd.e%d;e%fdZ{edddddddddddddddd d!d"d#d$Z|dFd%e#e!ejfd&e#e!ejfd'e#e!ejfd(e#e!e|fdMe#e!e|fd)e#e!e|fd*e#e!e|fd+e#e!e|fd,e#e!e|fd-e#e!e|fd.e%dNe%fdZ}edejNe}Z~d%e#e!ejfd&e#e!ejfd'e#e!ejfd(e#e!e|fdMe#e!e|fd)e#e!e|fd*e#e!e|fd+e#e!e|fd,e#e!e|fd-e#e!e|fd.e%dNe%fdZedddddddddddddddd d!d"d#d$ZdDd%e#e!ejfd&e#e!ejfd'e#e!ejfde#e!ejfd(e#e!efdMe#e!efd)e#e!efd*e#e!efd+e#e!efd,e#e!efd-e#e!efd.e%fdZedejNeZd%e#e!ejfd&e#e!ejfd'e#e!ejfde#e!ejfd(e#e!efdMe#e!efd)e#e!efd*e#e!efd+e#e!efd,e#e!efd-e#e!efd.e%fdZedddddddddddddddd d!d"d#d$ZdDd%e#e!ejfd&e#e!ejfd)e#e!efdSe#e!efdTe#e!efdUe#e!efd-e#e!efd.e%fdZedejNeZd%e#e!ejfd&e#e!ejfd)e#e!efdSe#e!efdTe#e!efdUe#e!efd-e#e!efd.e%fdZedddddddddddddddd d!d"d#d$ZdDd%e#e!ejfdZe#e!ejfd[e#e!ejfd\e#e!ejfd)e#e!efd6e#e!efd]e#e!efd,e#e!efd-e#e!efd.e%fdZedejNeZd%e#e!ejfdZe#e!ejfd[e#e!ejfd\e#e!ejfd)e#e!efd6e#e!efd]e#e!efd,e#e!efd-e#e!efd.e%fdZedddddddddddddddd d!d"d#d$ZdFd%e#e!ejfd4e#e!ejfdbe#e!ejfd-e#e!efd)e#e!efdBe#e!efdCe#e!efdce#e!efd.e%dde%fdZedejNeZd%e#e!ejfd4e#e!ejfdbe#e!ejfd-e#e!efd)e#e!efdBe#e!efdCe#e!efdce#e!efd.e%dde%fdZedddddddddddddddd d!d"d#d$ZdFd%e#e!ejfd4e#e!ejfdbe#e!ejfd-e#e!efd)e#e!efdBe#e!efdCe#e!efdie#e!efdce#e!efd.e%dde%fdZedejNeZd%e#e!ejfd4e#e!ejfdbe#e!ejfd-e#e!efd)e#e!efdBe#e!efdCe#e!efdie#e!efdce#e!efd.e%dde%fdZedddddddddddddddd d!d"d#d$ZdDd%e#e!ejfdSe#e!efdne#e!efd.e%fdZedejNeZd%e#e!ejfdSe#e!efdne#e!efd.e%fdZedddddddddddddddd d!d"d#d$ZdFd%e#e!ejfd4e#e!ejfd)e#e!efd-e#e!efd]e#e!efd.e%dNe%fdZedejNeZd%e#e!ejfd4e#e!ejfd)e#e!efd-e#e!efd]e#e!efd.e%dNe%fdZedddddddddddddddd d!d"d#d$ZdFd%e#e!ejfd4e#e!ejfd)e#e!efd-e#e!efd]e#e!efd.e%dNe%fdZedejNeZd%e#e!ejfd4e#e!ejfd)e#e!efd-e#e!efd]e#e!efd.e%dNe%fdZedddddddddddddddd d!d"d#d$ZdDd%e#e!ejfd&e#e!ejfd)e#e!efdwe#e!efdTe#e!efdUe#e!efd-e#e!efd.e%fdZed«ejNeZd%e#e!ejfd&e#e!ejfd)e#e!efdwe#e!efdTe#e!efdUe#e!efd-e#e!efd.e%fdÄZedddddddddddddddd d!d"d#d$ZdDd%e#e!ejfd4e#e!ejfd)e#e!efdBe#e!efdCe#e!efd-e#e!efd.e%fdńZedƫejNeZd%e#e!ejfd4e#e!ejfd)e#e!efdBe#e!efdCe#e!efd-e#e!efd.e%fdDŽZedddddddddddddddd d!d"d#d$ZdDd%e#e!ejfdSe#e!efdBe#e!efdCe#e!efdne#e!efd.e%f dɄZedʫejNeZd%e#e!ejfdSe#e!efdBe#e!efdCe#e!efdne#e!efd.e%f d˄Zedddddddddddddddd d!d"d#d$ZdDd%e#e!ejfd[e#e!ejfd\e#e!ejfd)e#e!efd6e#e!efd]e#e!efd,e#e!efd-e#e!efd.e%fd̈́ZedΫejNeZd%e#e!ejfd[e#e!ejfd\e#e!ejfd)e#e!efd6e#e!efd]e#e!efd,e#e!efd-e#e!efd.e%fdτZedddddddddddddddd d!d"d#d$ZedddZdDd%e#e!ejfd4e#e!ejfd5e#e!ejfd)e#e!efd6e#e!efd,e#e!efd-e#e!efde#e!efd.e%fdӄZedԫejNeZd%e#e!ejfd4e#e!ejfd5e#e!ejfd)e#e!efd6e#e!efd,e#e!efd-e#e!efde#e!efd.e%fdՄZedddddddddddddddd d!d"d#d$ZedddZdEd%e#e!ejfd4e#e!ejfd)e#e!efd-e#e!efde#e!efd.e%d;e%fd؄Zed٫ejNeZd%e#e!ejfd4e#e!ejfd)e#e!efd-e#e!efde#e!efd.e%d;e%fdڄZedddddddddddddddd d!d"d#d$ZedddZdDd%e#e!ejfd@e#e!ejfdAe#e!ejfd-e#e!efde#e!efd)e#e!efdBe#e!efdCe#e!efdDe#e!ejffd.e%fd݄ZedޫejNeZd%e#e!ejfd@e#e!ejfdAe#e!ejfd-e#e!efde#e!efd)e#e!efdBe#e!efdCe#e!efdDe#e!ejffd.e%fd߄Zedddddddddddddddd d!d"d#d$ZedddZdEd%e#e!ejfd4e#e!ejfd)e#e!efd,e#e!efd-e#e!efde#e!efd.e%d;e%fdZedejNeZd%e#e!ejfd4e#e!ejfd)e#e!efd,e#e!efd-e#e!efde#e!efd.e%d;e%fdZedddddddddddddddd d!d"d#d$ZedddZŐdDd%e#e!ejfdZe#e!ejfd[e#e!ejfd\e#e!ejfd)e#e!efd6e#e!efd]e#e!efd,e#e!efd-e#e!efde#e!efd.e%fdZedejNeƫZd%e#e!ejfdZe#e!ejfd[e#e!ejfd\e#e!ejfd)e#e!efd6e#e!efd]e#e!efd,e#e!efd-e#e!efde#e!efd.e%fdZedddddddddddddddd d!d"d#d$ZedddZʐdFd%e#e!ejfd4e#e!ejfdbe#e!ejfd-e#e!efde#e!efd)e#e!efdBe#e!efdCe#e!efdce#e!efd.e%dde%fdZedejNe˫Zd%e#e!ejfd4e#e!ejfdbe#e!ejfd-e#e!efde#e!efd)e#e!efdBe#e!efdCe#e!efdce#e!efd.e%dde%fdZedddddddddddddddd d!d"d#d$ZedddZϐdFd%e#e!ejfd4e#e!ejfdbe#e!ejfd-e#e!efde#e!efd)e#e!efdBe#e!efdCe#e!efdie#e!efdce#e!efd.e%dde%fdZedejNeЫZd%e#e!ejfd4e#e!ejfdbe#e!ejfd-e#e!efde#e!efd)e#e!efdBe#e!efdCe#e!efdie#e!efdce#e!efd.e%dde%fdZedddddddddddddddd d!d"d#d$ZedddZԐdFd%e#e!ejfd4e#e!ejfd)e#e!efd-e#e!efde#e!efd]e#e!efd.e%dNe%fdZedejNeիZd%e#e!ejfd4e#e!ejfd)e#e!efd-e#e!efde#e!efd]e#e!efd.e%dNe%fdZedddddddddddddddd d!d"d#d$ZedddZِdFd%e#e!ejfd4e#e!ejfd)e#e!efd-e#e!efde#e!efd]e#e!efd.e%dNe%fdZedejNeګZd%e#e!ejfd4e#e!ejfd)e#e!efd-e#e!efde#e!efd]e#e!efd.e%dNe%fdZedddddddddddddddd d!d"d#d$ZedddZސdDd%e#e!ejfd4e#e!ejfd)e#e!efdBe#e!efdCe#e!efd-e#e!efde#e!efd.e%fdZedejNe߫Zd%e#e!ejfd4e#e!ejfd)e#e!efdBe#e!efdCe#e!efd-e#e!efde#e!efd.e%fdZedddddddddddddddd d!d"d#d$ZedddZdDd%e#e!ejfdSe#e!efdBe#e!efdCe#e!efd-e#e!efde#e!efd.e%fdZedejNeZd%e#e!ejfdSe#e!efdBe#e!efdCe#e!efd-e#e!efde#e!efd.e%fdZedddddddddddddddd d!d"d#d$Zed ddZdDd%e#e!ejfd[e#e!ejfd\e#e!ejfd)e#e!efd6e#e!efd]e#e!efd,e#e!efd-e#e!efde#e!efd.e%fd Zed ejNeZd%e#e!ejfd[e#e!ejfd\e#e!ejfd)e#e!efd6e#e!efd]e#e!efd,e#e!efd-e#e!efde#e!efd.e%fd Zed ddddddddddddddd d!d"d#d$ZedddZdDd%e#e!efd4e#e!efd5e#e!efd)e#e!efd6e#e!efd,e#e!efd-e#e!efde#e!efd.e%d/e#e!effdZedejNeZd%e#e!efd4e#e!efd5e#e!efd)e#e!efd6e#e!efd,e#e!efd-e#e!efde#e!efd.e%d/e#e!effdZedddddddddddddddd d!d"d#d$ZedddZdEd%e#e!efd4e#e!efd)e#e!efd-e#e!efde#e!efd.e%d;e%d/e#e!effdZedejNeZd%e#e!efd4e#e!efd)e#e!efd-e#e!efde#e!efd.e%d;e%d/e#e!effdZedddddddddddddddd d!d"d#d$ZedddZdDd%e#e!efd@e#e!efdAe#e!efd-e#e!efde#e!efd)e#e!efdBe#e!efdCe#e!efdDe#e!ejffd.e%d/e#e!effdZedejNeZd%e#e!efd@e#e!efdAe#e!efd-e#e!efde#e!efd)e#e!efdBe#e!efdCe#e!efdDe#e!ejffd.e%d/e#e!effdZedddddddddddddddd d!d"d#d$ZedddZdEd%e#e!efd4e#e!efd)e#e!efd,e#e!efd-e#e!efde#e!efd.e%d;e%d/e#e!effdZedejNeZd%e#e!efd4e#e!efd)e#e!efd,e#e!efd-e#e!efde#e!efd.e%d;e%d/e#e!effd Zed!ddddddddddddddd d!d"d#d$Zed"ddZdDd%e#e!efdZe#e!efd[e#e!efd\e#e!efd)e#e!efd6e#e!efd]e#e!efd,e#e!efd-e#e!efde#e!efd.e%d/e#e!effd#Zed$ejNeZd%e#e!efdZe#e!efd[e#e!efd\e#e!efd)e#e!efd6e#e!efd]e#e!efd,e#e!efd-e#e!efde#e!efd.e%d/e#e!effd%Zed&ddddddddddddddd d!d"d#d$Zed'ddZdFd%e#e!efd4e#e!efdbe#e!efd-e#e!efde#e!efd)e#e!efdBe#e!efdCe#e!efdce#e!efd.e%dde%d/e#e!effd(Zed)ejNeZd%e#e!efd4e#e!efdbe#e!efd-e#e!efde#e!efd)e#e!efdBe#e!efdCe#e!efdce#e!efd.e%dde%d/e#e!effd*Z ed+ddddddddddddddd d!d"d#d$Z ed,ddZ dFd%e#e!e fd4e#e!e fdbe#e!e fd-e#e!e fde#e!e fd)e#e!e fdBe#e!e fdCe#e!e fdie#e!e fdce#e!e fd.e%dde%d/e#e!e ffd-Z ed.ejNe Z d%e#e!e fd4e#e!e fdbe#e!e fd-e#e!e fde#e!e fd)e#e!e fdBe#e!e fdCe#e!e fdie#e!e fdce#e!e fd.e%dde%d/e#e!e ffd/Zed0ddddddddddddddd d!d"d#d$Zed1ddZdFd%e#e!efd4e#e!efd)e#e!efd-e#e!efde#e!efd]e#e!efd.e%dNe%d/e#e!effd2Zed3ejNeZd%e#e!efd4e#e!efd)e#e!efd-e#e!efde#e!efd]e#e!efd.e%dNe%d/e#e!effd4Zed5ddddddddddddddd d!d"d#d$Zed6ddZdDd%e#e!efd4e#e!efd)e#e!efdBe#e!efdCe#e!efd-e#e!efde#e!efd.e%d/e#e!effd7Zed8ejNeZd%e#e!efd4e#e!efd)e#e!efdBe#e!efdCe#e!efd-e#e!efde#e!efd.e%d/e#e!effd9Zed:ddddddddddddddd d!d"d#d$Zed;ddZdDd%e#e!efdSe#e!efdBe#e!efdCe#e!efd-e#e!efde#e!efd.e%d/e#e!effd<Zed=ejNeZd%e#e!efdSe#e!efdBe#e!efdCe#e!efd-e#e!efde#e!efd.e%d/e#e!effd>Zed?ddddddddddddddd d!d"d#d$Zed@ddZdDd%e#e!efd[e#e!efd\e#e!efd)e#e!efd6e#e!efd]e#e!efd,e#e!efd-e#e!efde#e!efd.e%d/e#e!effdAZ edBejNe Z!d%e#e!efd[e#e!efd\e#e!efd)e#e!efd6e#e!efd]e#e!efd,e#e!efd-e#e!efde#e!efd.e%d/e#e!effdCZ"y(GzUPython wrappers around TensorFlow ops. This file is MACHINE GENERATED! Do not edit. N) pywrap_tfe)context)core)execute)dtypes)annotation_types)op_def_registry)ops)op_def_library)deprecated_endpoints)dispatch) tf_export)TypeVarListAny) AnnotatedTV_ApplyAdaMax_Tz_atypes.BFloat16z_atypes.Complex128z_atypes.Complex64z_atypes.Float32z_atypes.Float64z _atypes.Halfz _atypes.Int16z _atypes.Int32z _atypes.Int64z _atypes.Int8z_atypes.QInt16z_atypes.QInt32z _atypes.QInt8z_atypes.QUInt16z_atypes.QUInt8z_atypes.UInt16z_atypes.UInt32z_atypes.UInt64z _atypes.UInt8varmv beta1_powerlrbeta1beta2epsilongrad use_lockingreturnc tjxstj} | j} | jr t d| d} t j | d} tjd|||||||||| |  \} } }}|dd}t jrHd|jdd|jdf}|j}t jd||||\}|S)a/Update '*var' according to the AdaMax algorithm. m_t <- beta1 * m_{t-1} + (1 - beta1) * g v_t <- max(beta2 * v_{t-1}, abs(g)) variable <- variable - learning_rate / (1 - beta1^t) * m_t / (v_t + epsilon) Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). m: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). v: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). beta1_power: A `Tensor`. Must have the same type as `var`. Must be a scalar. lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. beta1: A `Tensor`. Must have the same type as `var`. Momentum factor. Must be a scalar. beta2: A `Tensor`. Must have the same type as `var`. Momentum factor. Must be a scalar. epsilon: A `Tensor`. Must have the same type as `var`. Ridge term. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var, m, and v tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Fapply_ada_max op does not support eager execution. Arg 'out' is a ref.NFr ApplyAdaMax rrrrrrrrrrnameT_contextr_thread_local_datais_eager RuntimeError_execute make_bool_op_def_library_apply_op_helpermust_record_gradient_get_attr_type_get_attr_boolinputsrecord_gradient)rrrrrrrrrrr#_ctxtld__op_outputs_result_attrs _inputs_flats V/home/dcms/DCMS/lib/python3.12/site-packages/tensorflow/python/ops/gen_training_ops.py apply_ada_maxr<sD    0h..0$ #\\ _ ``K""; >+'883!qkb"%t#.T;!QX QK' ""$3%%c*M  /1F::L |VW6 (' .zraw_ops.ApplyAdaMaxc td)Nr r)) rrrrrrrrrrr#ctxs r;apply_ada_max_eager_fallbackrAV]^^r=TV_ApplyAdadelta_Taccum accum_updaterhoc tjxstj} | j} | jr t d|d}t j |d}tjd||||||||| \} } } } | dd}t jrHd| jdd| jdf}| j}t jd||||\}|S)aUpdate '*var' according to the adadelta scheme. accum = rho() * accum + (1 - rho()) * grad.square(); update = (update_accum + epsilon).sqrt() * (accum + epsilon()).rsqrt() * grad; update_accum = rho() * update_accum + (1 - rho()) * update.square(); var -= update; Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). accum: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). accum_update: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. rho: A `Tensor`. Must have the same type as `var`. Decay factor. Must be a scalar. epsilon: A `Tensor`. Must have the same type as `var`. Constant factor. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If True, updating of the var, accum and update_accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Gapply_adadelta op does not support eager execution. Arg 'out' is a ref.NFr ApplyAdadelta rrDrErrFrrrr#r$r%)rrDrErrFrrrr#r3r4r5r6r7r8r9r:s r;apply_adadeltarK[s<    0h..0$ #\\ ` aaK""; >+'88SLCt%0t=!QX QK' ""$3%%c*M  /1F::L vw8 (' .r=zraw_ops.ApplyAdadeltac td)NrHr?) rrDrErrFrrrr#r@s r;apply_adadelta_eager_fallbackrM^__r=TV_ApplyAdagrad_T update_slotsc 0tjxstj}|j}|jr t d|d}t j |d}|d}t j |d}tjd|||||||\} } } } | dd} t jrYd | jd d| jdd| jdf} | j}t jd|| | | \} | S) a/Update '*var' according to the adagrad scheme. accum += grad * grad var -= lr * grad * (1 / sqrt(accum)) Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). accum: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. update_slots: An optional `bool`. Defaults to `True`. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Fapply_adagrad op does not support eager execution. Arg 'out' is a ref.NFrTrP ApplyAdagradrrDrrrrPr#r$r%)rrDrrrrPr#r3r4r5r6r7r8r9r:s r; apply_adagradrUs0    0h..0$ #\\ _ ``K""; >+L##L.A,'88Cu$$/l!#!QX QK' ""$3%%c*M  /  02F::L  fg7 (' .r=zraw_ops.ApplyAdagradctd)NrRr?)rrDrrrrPr#r@s r;apply_adagrad_eager_fallbackrWrBr=TV_ApplyAdagradDA_Tgradient_accumulatorgradient_squared_accumulatorl1l2 global_stepc tjxstj} | j} | jr t d|d}t j |d}tjd||||||||||  \} } } }|dd}t jrHd| jdd| jdf}| j}t jd||||\}|S)a^Update '*var' according to the proximal adagrad scheme. Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). gradient_accumulator: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). gradient_squared_accumulator: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). grad: A `Tensor`. Must have the same type as `var`. The gradient. lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. l1: A `Tensor`. Must have the same type as `var`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `var`. L2 regularization. Must be a scalar. global_step: A `Tensor` of type `int64`. Training step number. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If True, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Iapply_adagrad_da op does not support eager execution. Arg 'out' is a ref.NFrApplyAdagradDA rrYrZrrr[r\r]rr#r$r%)rrYrZrrr[r\r]rr#r3r4r5r6r7r8r9r:s r;apply_adagrad_darbs6    0h..0$ #\\ b ccK""; >+'88c8L7S#rb&1{# %!QX QK' ""$3%%c*M  /1F::L ,9 (' .r=zraw_ops.ApplyAdagradDAc td)Nr_r?) rrYrZrrr[r\r]rr#r@s r;apply_adagrad_da_eager_fallbackrd`aar=TV_ApplyAdagradV2_Tc 2tjxstj}|j} | jr t d|d}t j |d}|d}t j |d}tjd|||||||| \} } } } | dd} t jrYd | jd d| jdd| jdf}| j}t jd||| | \} | S) aUpdate '*var' according to the adagrad scheme. accum += grad * grad var -= lr * grad * (1 / sqrt(accum)) Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). accum: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. epsilon: A `Tensor`. Must have the same type as `var`. Constant factor. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. update_slots: An optional `bool`. Defaults to `True`. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Iapply_adagrad_v2 op does not support eager execution. Arg 'out' is a ref.NFrTrPApplyAdagradV2rrDrrrrrPr#r$r%)rrDrrrrrPr#r3r4r5r6r7r8r9r:s r;apply_adagrad_v2rk s 4    0h..0$ #\\ b ccK""; >+L##L.A,'88c2w#'3$@!QX QK' ""$3%%c*M  /  02F::L ,9 (' .r=zraw_ops.ApplyAdagradV2c td)Nrhr?) rrDrrrrrPr#r@s r;apply_adagrad_v2_eager_fallbackrmCrer=TV_ApplyAdam_T beta2_power use_nesterovc <tjxstj} | j}|jr t d| d} t j | d} | d} t j | d} tjd|||||||||| | | | \}}}}|dd}t jrYd|jdd|jdd|jdf}|j}t jd||||\}|S) a Update '*var' according to the Adam algorithm. $$\text{lr}_t := \mathrm{lr} \cdot \frac{\sqrt{1 - \beta_2^t}}{1 - \beta_1^t}$$ $$m_t := \beta_1 \cdot m_{t-1} + (1 - \beta_1) \cdot g$$ $$v_t := \beta_2 \cdot v_{t-1} + (1 - \beta_2) \cdot g^2$$ $$\text{var} := \begin{cases} \text{var} - (m_t \beta_1 + g \cdot (1 - \beta_1))\cdot\text{lr}_t/(\sqrt{v_t} + \epsilon), &\text{if use_nesterov}\\\\ \text{var} - m_t \cdot \text{lr}_t /(\sqrt{v_t} + \epsilon), &\text{otherwise} \end{cases}$$ Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). m: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). v: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). beta1_power: A `Tensor`. Must have the same type as `var`. Must be a scalar. beta2_power: A `Tensor`. Must have the same type as `var`. Must be a scalar. lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. beta1: A `Tensor`. Must have the same type as `var`. Momentum factor. Must be a scalar. beta2: A `Tensor`. Must have the same type as `var`. Momentum factor. Must be a scalar. epsilon: A `Tensor`. Must have the same type as `var`. Ridge term. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var, m, and v tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. use_nesterov: An optional `bool`. Defaults to `False`. If `True`, uses the nesterov update. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Capply_adam op does not support eager execution. Arg 'out' is a ref.NFrrp ApplyAdam rrrrrorrrrrrrpr#r$r%)rrrrrorrrrrrrpr#r3r4r5r6r7r8r9r:s r; apply_adamruHs+N    0h..0$ #\\ \ ]]K""; >+L##L.A,'88QK!,5$4[".T ;!QX QK' ""$3%%c*M  /  02F::L \674 (' .r=zraw_ops.ApplyAdamctd)Nrrr?)rrrrrorrrrrrrpr#r@s r;apply_adam_eager_fallbackrwZ[[r=TV_ApplyAddSign_Talpha sign_decaybetac tjxstj} | j} | jr t d|d}t j |d}tjd||||||||| \} } } } | dd}t jrHd| jdd| jdf}| j}t jd||||\}|S)aUpdate '*var' according to the AddSign update. m_t <- beta1 * m_{t-1} + (1 - beta1) * g update <- (alpha + sign_decay * sign(g) *sign(m)) * g variable <- variable - lr_t * update Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). m: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. alpha: A `Tensor`. Must have the same type as `var`. Must be a scalar. sign_decay: A `Tensor`. Must have the same type as `var`. Must be a scalar. beta: A `Tensor`. Must have the same type as `var`. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and m tensors is protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Gapply_add_sign op does not support eager execution. Arg 'out' is a ref.NFr ApplyAddSign rrrrzr{r|rrr#r$r%)rrrrzr{r|rrr#r3r4r5r6r7r8r9r:s r;apply_add_signrs8    0h..0$ #\\ ` aaK""; >+'88C15#-Dt$/d<!QX QK' ""$3%%c*M  /1F::L  fg7 (' .r=zraw_ops.ApplyAddSignc td)Nr~r?) rrrrzr{r|rrr#r@s r;apply_add_sign_eager_fallbackrrNr=TV_ApplyCenteredRMSProp_Tmgmsmommomentumc tjxstj} | j} | jr t d| d} t j | d} tjd|||||||||| |  \} } }}|dd}t jrHd|jdd|jdf}|j}t jd||||\}|S)a/ Update '*var' according to the centered RMSProp algorithm. The centered RMSProp algorithm uses an estimate of the centered second moment (i.e., the variance) for normalization, as opposed to regular RMSProp, which uses the (uncentered) second moment. This often helps with training, but is slightly more expensive in terms of computation and memory. Note that in dense implementation of this algorithm, mg, ms, and mom will update even if the grad is zero, but in this sparse implementation, mg, ms, and mom will not update in iterations during which the grad is zero. mean_square = decay * mean_square + (1-decay) * gradient ** 2 mean_grad = decay * mean_grad + (1-decay) * gradient Delta = learning_rate * gradient / sqrt(mean_square + epsilon - mean_grad ** 2) mg <- rho * mg_{t-1} + (1-rho) * grad ms <- rho * ms_{t-1} + (1-rho) * grad * grad mom <- momentum * mom_{t-1} + lr * grad / sqrt(ms - mg * mg + epsilon) var <- var - mom Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). mg: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). ms: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). mom: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. rho: A `Tensor`. Must have the same type as `var`. Decay rate. Must be a scalar. momentum: A `Tensor`. Must have the same type as `var`. Momentum Scale. Must be a scalar. epsilon: A `Tensor`. Must have the same type as `var`. Ridge term. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var, mg, ms, and mom tensors is protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Papply_centered_rms_prop op does not support eager execution. Arg 'out' is a ref.NFrApplyCenteredRMSProp rrrrrrFrrrrr#r$r%)rrrrrrFrrrrr#r3r4r5r6r7r8r9r:s r;apply_centered_rms_proprsb    0h..0$ #\\ i jjK""; >+'88CB232$'(G%){O!QX QK' ""$3%%c*M  /1F::L  fg? (' .r=zraw_ops.ApplyCenteredRMSPropc td)Nrr?) rrrrrrFrrrrr#r@s r;&apply_centered_rms_prop_eager_fallbackrghhr=TV_ApplyFtrl_Tlinearlr_powermultiply_linear_by_lrc 8tjxstj} | j} | jr t d|d}t j |d}| d} t j | d} tjd|||||||||| |  \} } }}|dd}t jrYd|jdd|jdd|jdf}|j}t jd||||\}|S) aUpdate '*var' according to the Ftrl-proximal scheme. accum_new = accum + grad * grad linear += grad - (accum_new^(-lr_power) - accum^(-lr_power)) / lr * var quadratic = 1.0 / (accum_new^(lr_power) * lr) + 2 * l2 var = (sign(linear) * l1 - linear) / quadratic if |linear| > l1 else 0.0 accum = accum_new Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). accum: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). linear: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). grad: A `Tensor`. Must have the same type as `var`. The gradient. lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. l1: A `Tensor`. Must have the same type as `var`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `var`. L2 regularization. Must be a scalar. lr_power: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. multiply_linear_by_lr: An optional `bool`. Defaults to `False`. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Capply_ftrl op does not support eager execution. Arg 'out' is a ref.NFrr ApplyFtrl rrDrrrr[r\rrrr#r$r%)rrDrrrr[r\rrrr#r3r4r5r6r7r8r9r:s r; apply_ftrlrs-F    0h..0$ #\\ \ ]]K""; >+"!",,-BD[\'88E&trH++@tM!QX QK' ""$3%%c*M  /1H  !89;F::L \674 (' .r=zraw_ops.ApplyFtrlc td)Nrr?) rrDrrrr[r\rrrr#r@s r;apply_ftrl_eager_fallbackr[rxr=TV_ApplyFtrlV2_T l2_shrinkagec :tjxstj} | j} | jr t d| d} t j | d} | d} t j | d} tjd|||||||||| | |  \}}}}|dd}t jrYd|jdd|jdd|jdf}|j}t jd||||\}|S) a%Update '*var' according to the Ftrl-proximal scheme. grad_with_shrinkage = grad + 2 * l2_shrinkage * var accum_new = accum + grad * grad linear += grad_with_shrinkage - (accum_new^(-lr_power) - accum^(-lr_power)) / lr * var quadratic = 1.0 / (accum_new^(lr_power) * lr) + 2 * l2 var = (sign(linear) * l1 - linear) / quadratic if |linear| > l1 else 0.0 accum = accum_new Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). accum: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). linear: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). grad: A `Tensor`. Must have the same type as `var`. The gradient. lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. l1: A `Tensor`. Must have the same type as `var`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `var`. L2 shrinkage regularization. Must be a scalar. l2_shrinkage: A `Tensor`. Must have the same type as `var`. lr_power: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. multiply_linear_by_lr: An optional `bool`. Defaults to `False`. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Fapply_ftrl_v2 op does not support eager execution. Arg 'out' is a ref.NFrr ApplyFtrlV2 rrDrrrr[r\rrrrr#r$r%)rrDrrrr[r\rrrrr#r3r4r5r6r7r8r9r:s r; apply_ftrl_v2r`s0L    0h..0$ #\\ _ ``K""; >+"!",,-BD[\'883eF", (k-B O!QX QK' ""$3%%c*M  /1H  !89;F::L |VW6 (' .r=zraw_ops.ApplyFtrlV2c td)Nrr?) rrDrrrr[r\rrrrr#r@s r;apply_ftrl_v2_eager_fallbackrrBr=TV_ApplyGradientDescent_Tdeltactjxstj}|j}|jr t d|d}t j |d}tjd|||||\}}}} | dd} t jrHd|jdd|jdf} |j} t jd| | | | \} | S)aAUpdate '*var' by subtracting 'alpha' * 'delta' from it. Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). alpha: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. delta: A `Tensor`. Must have the same type as `var`. The change. use_locking: An optional `bool`. Defaults to `False`. If `True`, the subtraction will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Oapply_gradient_descent op does not support eager execution. Arg 'out' is a ref.NFrApplyGradientDescentrrzrrr#r$r%) rrzrrr#r3r4r5r6r7r8r9r:s r;apply_gradient_descentrs"    0h..0$ #\\ h iiK""; >+'88CuE,7dD!QX QK' ""$3%%c*M  /1F::L  fg? (' .r=zraw_ops.ApplyGradientDescentctd)Nrr?)rrzrrr#r@s r;%apply_gradient_descent_eager_fallbackrfggr=TV_ApplyMomentum_Tc 2tjxstj}|j} | jr t d|d}t j |d}|d}t j |d}tjd|||||||| \} } } } | dd} t jrYd| jdd| jdd| jdf}| j}t jd||| | \} | S) atUpdate '*var' according to the momentum scheme. Set use_nesterov = True if you want to use Nesterov momentum. accum = accum * momentum + grad var -= lr * accum Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). accum: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. momentum: A `Tensor`. Must have the same type as `var`. Momentum. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. use_nesterov: An optional `bool`. Defaults to `False`. If `True`, the tensor passed to compute grad will be var - lr * momentum * accum, so in the end, the var you get is actually var - lr * momentum * accum. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Gapply_momentum op does not support eager execution. Arg 'out' is a ref.NFrrp ApplyMomentumrrDrrrrrpr#r$r%)rrDrrrrrpr#r3r4r5r6r7r8r9r:s r;apply_momentumrs>    0h..0$ #\\ ` aaK""; >+L##L.A,'88S"4"* &2?!QX QK' ""$3%%c*M  /  02F::L vw8 (' .r=zraw_ops.ApplyMomentumc td)Nrr?) rrDrrrrrpr#r@s r;apply_momentum_eager_fallbackrrNr=TV_ApplyPowerSign_Tlogbasec tjxstj} | j} | jr t d|d}t j |d}tjd||||||||| \} } } } | dd}t jrHd| jdd| jdf}| j}t jd||||\}|S)a)Update '*var' according to the AddSign update. m_t <- beta1 * m_{t-1} + (1 - beta1) * g update <- exp(logbase * sign_decay * sign(g) * sign(m_t)) * g variable <- variable - lr_t * update Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). m: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. logbase: A `Tensor`. Must have the same type as `var`. Must be a scalar. sign_decay: A `Tensor`. Must have the same type as `var`. Must be a scalar. beta: A `Tensor`. Must have the same type as `var`. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and m tensors is protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Iapply_power_sign op does not support eager execution. Arg 'out' is a ref.NFrApplyPowerSign rrrrr{r|rrr#r$r%)rrrrr{r|rrr#r3r4r5r6r7r8r9r:s r;apply_power_signrs8    0h..0$ #\\ b ccK""; >+'88cQ2w%/d&1>!QX QK' ""$3%%c*M  /1F::L ,9 (' .r=zraw_ops.ApplyPowerSignc td)Nrr?) rrrrr{r|rrr#r@s r;apply_power_sign_eager_fallbackrMrer=TV_ApplyProximalAdagrad_Tc tjxstj}|j} | jr t d|d}t j |d}tjd|||||||| \} } } } | dd} t jrHd| jdd| jdf}| j}t jd||| | \} | S)a Update '*var' and '*accum' according to FOBOS with Adagrad learning rate. accum += grad * grad prox_v = var - lr * grad * (1 / sqrt(accum)) var = sign(prox_v)/(1+lr*l2) * max{|prox_v|-lr*l1,0} Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). accum: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. l1: A `Tensor`. Must have the same type as `var`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `var`. L2 regularization. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If True, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Oapply_proximal_adagrad op does not support eager execution. Arg 'out' is a ref.NFrApplyProximalAdagradrrDrr[r\rrr#r$r%)rrDrr[r\rrr#r3r4r5r6r7r8r9r:s r;apply_proximal_adagradrRs6    0h..0$ #\\ h iiK""; >+'88Cur%){O!QX QK' ""$3%%c*M  /1F::L  fg? (' .r=zraw_ops.ApplyProximalAdagradc td)Nrr?) rrDrr[r\rrr#r@s r;%apply_proximal_adagrad_eager_fallbackrrr=!TV_ApplyProximalGradientDescent_Tc tjxstj}|j}|jr t d|d}t j |d}tjd|||||||\} } } } | dd} t jrHd| jdd| jdf} | j}t jd|| | | \} | S)aaUpdate '*var' as FOBOS algorithm with fixed learning rate. prox_v = var - alpha * delta var = sign(prox_v)/(1+alpha*l2) * max{|prox_v|-alpha*l1,0} Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). alpha: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. l1: A `Tensor`. Must have the same type as `var`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `var`. L2 regularization. Must be a scalar. delta: A `Tensor`. Must have the same type as `var`. The change. use_locking: An optional `bool`. Defaults to `False`. If True, the subtraction will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Xapply_proximal_gradient_descent op does not support eager execution. Arg 'out' is a ref.NFrApplyProximalGradientDescentrrzr[r\rrr#r$r%)rrzr[r\rrr#r3r4r5r6r7r8r9r:s r;apply_proximal_gradient_descentrs0    0h..0$ #\\ q rrK""; >+'88&Cu.3-13!QX QK' ""$3%%c*M  /1F::L & fgG (' .r=z$raw_ops.ApplyProximalGradientDescentctd)Nrr?)rrzr[r\rrr#r@s r;.apply_proximal_gradient_descent_eager_fallbackrsoppr=TV_ApplyRMSProp_Tc tjxstj} | j} | jr t d|d}t j |d}tjd||||||||||  \} } } }|dd}t jrHd| jdd| jdf}| j}t jd||||\}|S)aUpdate '*var' according to the RMSProp algorithm. Note that in dense implementation of this algorithm, ms and mom will update even if the grad is zero, but in this sparse implementation, ms and mom will not update in iterations during which the grad is zero. mean_square = decay * mean_square + (1-decay) * gradient ** 2 Delta = learning_rate * gradient / sqrt(mean_square + epsilon) ms <- rho * ms_{t-1} + (1-rho) * grad * grad mom <- momentum * mom_{t-1} + lr * grad / sqrt(ms + epsilon) var <- var - mom Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). ms: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). mom: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. rho: A `Tensor`. Must have the same type as `var`. Decay rate. Must be a scalar. momentum: A `Tensor`. Must have the same type as `var`. epsilon: A `Tensor`. Must have the same type as `var`. Ridge term. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var, ms, and mom tensors is protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Gapply_rms_prop op does not support eager execution. Arg 'out' is a ref.NFr ApplyRMSProp rrrrrFrrrrr#r$r%)rrrrrFrrrrr#r3r4r5r6r7r8r9r:s r;apply_rms_proprsL    0h..0$ #\\ ` aaK""; >+'88CBCBC!)7$/d<!QX QK' ""$3%%c*M  /1F::L  fg7 (' .r=zraw_ops.ApplyRMSPropc td)Nrr?) rrrrrFrrrrr#r@s r;apply_rms_prop_eager_fallbackrrNr=TV_ResourceApplyAdaMax_Tc Btjxstj} | j} | jr% t j | d| |||||||||d| } | S| d} tj| d} tj d|||||||||| |  \}}}}|S#t j$r }tj|| Yd}~nd}~wt j$rYnwxYw t|||||||||| | |  S#t j$rYwxYw)aUpdate '*var' according to the AdaMax algorithm. m_t <- beta1 * m_{t-1} + (1 - beta1) * g v_t <- max(beta2 * v_{t-1}, abs(g)) variable <- variable - learning_rate / (1 - beta1^t) * m_t / (v_t + epsilon) Args: var: A `Tensor` of type `resource`. Should be from a Variable(). m: A `Tensor` of type `resource`. Should be from a Variable(). v: A `Tensor` of type `resource`. Should be from a Variable(). beta1_power: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Must be a scalar. lr: A `Tensor`. Must have the same type as `beta1_power`. Scaling factor. Must be a scalar. beta1: A `Tensor`. Must have the same type as `beta1_power`. Momentum factor. Must be a scalar. beta2: A `Tensor`. Must have the same type as `beta1_power`. Momentum factor. Must be a scalar. epsilon: A `Tensor`. Must have the same type as `beta1_power`. Ridge term. Must be a scalar. grad: A `Tensor`. Must have the same type as `beta1_power`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var, m, and v tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: The created Operation. ResourceApplyAdaMaxrNrr#r@Fr")r&rr'r(rTFE_Py_FastPathExecute_core_NotOkStatusException_opsraise_from_not_ok_status_FallbackException%resource_apply_ada_max_eager_fallback_SymbolicExceptionr*r+r,r-)rrrrrrrrrrr#r3r4r8er5r6r7s r;resource_apply_ada_maxrsH>    0h..0$ #\\ 11 #T31k2u wm[:gnK""; >+'883!qk"$E'.T+6T C!QX *'  & &- ##At,,  # #    2 q!["eUGT!$88  # #   0#B&&C-9CC-,C-1DDDzraw_ops.ResourceApplyAdaMaxc | d} tj| d} tj||||||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}}}}t/j0|tj2}t/j0|tj2}t/j0|tj2}|||||||||g }d| d| f}tj4dd||| | }d}|S)NFrr$sResourceApplyAdaMaxrr1attrsr@r#r*r+args_to_matching_eager_dtypesfloat32float64int32uint8int16int8 complex64int64qint8quint8qint32bfloat16qint16quint16uint16 complex128halfuint32uint64rconvert_to_tensorresourcer)rrrrrrrrrrr#r@_attr_T _inputs_Tr:r9r8s r;rrBs|K""; >+66 RPUW^`d7egjmtm|m|FNNPWP]P]_f_l_lnun{n{}D}I}IKRK\K\^e^k^kmtmzmz|C|J|JLSLZLZ\c\l\lnun|n|~E~M~MOVO]O]_f_q_qszssAHAOAOQXQ_Q_mbc'93<0;E5'4 sG$4$45# Q 0 01! Q 0 01!q!["eUGTJ, - 5&   3Q|#)s ?' ' .r=TV_ResourceApplyAdadelta_Tc 6tjxstj} | j} | jr# t j | d||||||||d| } | S|d}tj|d}tj d||||||||| \} } }}|S#t j$r } tj| |Yd} ~ nd} ~ wt j$rYnwxYw t||||||||||  S#t j$rYwxYw)aWUpdate '*var' according to the adadelta scheme. accum = rho() * accum + (1 - rho()) * grad.square(); update = (update_accum + epsilon).sqrt() * (accum + epsilon()).rsqrt() * grad; update_accum = rho() * update_accum + (1 - rho()) * update.square(); var -= update; Args: var: A `Tensor` of type `resource`. Should be from a Variable(). accum: A `Tensor` of type `resource`. Should be from a Variable(). accum_update: A `Tensor` of type `resource`. Should be from a Variable(). lr: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Scaling factor. Must be a scalar. rho: A `Tensor`. Must have the same type as `lr`. Decay factor. Must be a scalar. epsilon: A `Tensor`. Must have the same type as `lr`. Constant factor. Must be a scalar. grad: A `Tensor`. Must have the same type as `lr`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If True, updating of the var, accum and update_accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: The created Operation. ResourceApplyAdadeltarNrFrJ)r&rr'r(rrrrrrr&resource_apply_adadelta_eager_fallbackrr*r+r,r-)rrDrErrFrrrr#r3r4r8rr5r6r7s r;resource_apply_adadeltarUs<6    0h..0$ #\\ 11 %tS%r WdM;8gnK""; >+'88S.:rs)0t-8t E!QX *'  & &- ##At,,  # #    3 ulBWd!$88  # #   0!B""C)5CC)(C)-DDDzraw_ops.ResourceApplyAdadeltac |d}tj|d}tj||||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}}t/j0|tj2}t/j0|tj2}t/j0|tj2}|||||||g} d| d|f} tj4dd| | | |}d}|S)NFrr$sResourceApplyAdadeltarrr)rrDrErrFrrrr#r@rrr:r9r8s r;rrsaK""; >+66C$7OQTW^WfWfhohwhwzAzGzGIPIVIVX_XeXegngsgsu|uFuFHOHUHUW^WdWdfmftftv}vDvDFMFVFVX_XfXfhohwhwy@yGyGIPI[I[]d]i]ikrkyky{B{I{IWLM'9&2sGT sG$4$45#  (8(8 9%'' g6F6FG,ulBWdC, - 5&   5q#)s ?' ' .r=TV_ResourceApplyAdagrad_Tc `tjxstj}|j}|jr" t j |d|||||d|d| } | S|d}tj|d}|d}tj|d}tj d|||||||\} } } } | S#t j$r } tj| |Yd} ~ nd} ~ wt j$rYnwxYw t||||||||S#t j$rYwxYw) aUpdate '*var' according to the adagrad scheme. accum += grad * grad var -= lr * grad * (1 / sqrt(accum)) Args: var: A `Tensor` of type `resource`. Should be from a Variable(). accum: A `Tensor` of type `resource`. Should be from a Variable(). lr: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Scaling factor. Must be a scalar. grad: A `Tensor`. Must have the same type as `lr`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. update_slots: An optional `bool`. Defaults to `True`. name: A name for the operation (optional). Returns: The created Operation. ResourceApplyAdagradrrPNrrPr#r@FTrT)r&rr'r(rrrrrrr%resource_apply_adagrad_eager_fallbackrr*r+r,r-)rrDrrrrPr#r3r4r8rr5r6r7s r;resource_apply_adagradrsO,    0h..0$ #\\ 11 $dCD{NLBgnK""; >+L##L.A,'88Cu$,7-9F!QX *+  & &- ##At,,  # #    2 ub$K#$D::  # #   s0 B99D C''D?DDD-,D-zraw_ops.ResourceApplyAdagradc|d}tj|d}|d}tj|d}tj||g|tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\}} | \}}t/j0|tj2}t/j0|tj2}||||g} d|d|d|f} tj4dd| | ||} d} | S) NFrTrPr$sResourceApplyAdagradrrr) rrDrrrrPr#r@rrr:r9r8s r;rrsMK""; >+L##L.A,66Dz3ZaZiZikrkxkx{B{H{HJQJWJWY`YeYegngxgxzAzGzGIPIVIVX_XfXfhohvhvxxHxHJQJXJXZaZiZikrkyky{B{M{MOVO[O[]d]k]kmtm{m{I~'9*2t sG$4$45#  (8(8 9%ub$', -n &   4a #)s ?' ' .r=TV_ResourceApplyAdagradDA_Tc <tjxstj} | j} | jr$ t j | d| ||||||||d| } | S|d}tj|d}tj d||||||||||  \}}}}|S#t j$r } tj| | Yd} ~ nd} ~ wt j$rYnwxYw t|||||||||| |  S#t j$rYwxYw)aUpdate '*var' according to the proximal adagrad scheme. Args: var: A `Tensor` of type `resource`. Should be from a Variable(). gradient_accumulator: A `Tensor` of type `resource`. Should be from a Variable(). gradient_squared_accumulator: A `Tensor` of type `resource`. Should be from a Variable(). grad: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. The gradient. lr: A `Tensor`. Must have the same type as `grad`. Scaling factor. Must be a scalar. l1: A `Tensor`. Must have the same type as `grad`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `grad`. L2 regularization. Must be a scalar. global_step: A `Tensor` of type `int64`. Training step number. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If True, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: The created Operation. ResourceApplyAdagradDArNrFra)r&rr'r(rrrrrrr(resource_apply_adagrad_da_eager_fallbackrr*r+r,r-)rrYrZrrr[r\r]rr#r3r4r8rr5r6r7s r;resource_apply_adagrad_darsJ6    0h..0$ #\\  11 &c3G$dBB {$gnK""; >+'88 c7K?['+rb.9.9 F!QX *+  & &- ##At,,  # #    5 #%A4 b+;TtMM  # #   0"B$$C+7CC+*C+/DDDzraw_ops.ResourceApplyAdagradDAc 8|d}tj|d}tj||||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}}t/j0|tj2}t/j0|tj2}t/j0|tj2}t/j0|tj}||||||||g} d| d|f}tj4dd| || | }d}|S)NFrr$sResourceApplyAdagradDArrr)rrYrZrrr[r\r]rr#r@rrr:r9r8s r;rr,sK""; >+66b"b7I3QXQ`Q`bibqbqsztAtACJCPCPRYR_R_ahamamovo@o@BIBOBOQXQ^Q^`g`n`npwp~p~@G@P@PRYR`R`bibqbqszsAsACJCUCUW^WcWcelesesu|uCuCQFG'9 4R sG$4$45#//0DgFVFVW!%!7!78TV]VfVf!g&&{GMMB++-I4QSUWY[]hi, - 5&   6$0C"& (' ' .r=TV_ResourceApplyAdagradV2_Tcftjxstj}|j} | jr# t j |d||||||d|d| } | S|d}tj|d}|d}tj|d}tj d|||||||| \} } } }| S#t j$r } tj| |Yd} ~ nd} ~ wt j$rYnwxYw t||||||||| S#t j$rYwxYw) aJUpdate '*var' according to the adagrad scheme. accum += grad * grad var -= lr * grad * (1 / (sqrt(accum) + epsilon)) Args: var: A `Tensor` of type `resource`. Should be from a Variable(). accum: A `Tensor` of type `resource`. Should be from a Variable(). lr: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Scaling factor. Must be a scalar. epsilon: A `Tensor`. Must have the same type as `lr`. Constant factor. Must be a scalar. grad: A `Tensor`. Must have the same type as `lr`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. update_slots: An optional `bool`. Defaults to `True`. name: A name for the operation (optional). Returns: The created Operation. ResourceApplyAdagradV2rrPNrFTrj)r&rr'r(rrrrrrr(resource_apply_adagrad_v2_eager_fallbackrr*r+r,r-)rrDrrrrrPr#r3r4r8rr5r6r7s r;resource_apply_adagrad_v2rAsV0    0h..0$ #\\ 11 &c5"gt{NLBgnK""; >+L##L.A,'88 c2*1.9/;$ H!QX *-  & &- ##At,,  # #    5 ub'4[#$D::  # #   0!B;;DC))DDDD0/D0zraw_ops.ResourceApplyAdagradV2c |d}tj|d}|d}tj|d}tj|||g|tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}t/j0|tj2}t/j0|tj2}|||||g} d| d|d|f} tj4dd| | ||} d} | S) NFrTrPr$sResourceApplyAdagradV2rrr)rrDrrrrrPr#r@rrr:r9r8s r;rr{scK""; >+L##L.A,66GT7JCRYRaRacjcrcrt{uBuBDKDQDQSZS`S`bibnbnpwpApACJCPCPRYR_R_ahaoaoqxqqAHAQAQSZSaSacjcrcrt{tBtBDKDVDVX_XdXdfmftftv}vDvDRGH'9!2w sG$4$45#  (8(8 9%ub'40, -n &   6$0C"& (' ' .r=TV_ResourceApplyAdam_Tc tjxstj} | j}|jr( t j | d| |||||||||| d| d| }|S| d} tj| d} | d} tj| d} tj d|||||||||| | | | \}}}}|S#t j$r }tj|| Yd}~nd}~wt j$rYnwxYw t|||||||||| | | | | S#t j$rYwxYw)aUpdate '*var' according to the Adam algorithm. $$\text{lr}_t := \mathrm{lr} \cdot \frac{\sqrt{1 - \beta_2^t}}{1 - \beta_1^t}$$ $$m_t := \beta_1 \cdot m_{t-1} + (1 - \beta_1) \cdot g$$ $$v_t := \beta_2 \cdot v_{t-1} + (1 - \beta_2) \cdot g^2$$ $$\text{var} := \begin{cases} \text{var} - (m_t \beta_1 + g \cdot (1 - \beta_1))\cdot\text{lr}_t/(\sqrt{v_t} + \epsilon), &\text{if use_nesterov}\\\\ \text{var} - m_t \cdot \text{lr}_t /(\sqrt{v_t} + \epsilon), &\text{otherwise} \end{cases}$$ Args: var: A `Tensor` of type `resource`. Should be from a Variable(). m: A `Tensor` of type `resource`. Should be from a Variable(). v: A `Tensor` of type `resource`. Should be from a Variable(). beta1_power: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Must be a scalar. beta2_power: A `Tensor`. Must have the same type as `beta1_power`. Must be a scalar. lr: A `Tensor`. Must have the same type as `beta1_power`. Scaling factor. Must be a scalar. beta1: A `Tensor`. Must have the same type as `beta1_power`. Momentum factor. Must be a scalar. beta2: A `Tensor`. Must have the same type as `beta1_power`. Momentum factor. Must be a scalar. epsilon: A `Tensor`. Must have the same type as `beta1_power`. Ridge term. Must be a scalar. grad: A `Tensor`. Must have the same type as `beta1_power`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var, m, and v tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. use_nesterov: An optional `bool`. Defaults to `False`. If `True`, uses the nesterov update. name: A name for the operation (optional). Returns: The created Operation. ResourceApplyAdamrrpNrrpr#r@Frt)r&rr'r(rrrrrrr"resource_apply_adam_eager_fallbackrr*r+r,r-)rrrrrorrrrrrrpr#r3r4r8rr5r6r7s r;resource_apply_adamr%swH    0h..0$ #\\  11 !4aK E5'4  &gnK""; >+L##L.A,'88QK)45#(')4*6T C!QX *1  & &- ##At,,  # #    / q![+r5% Kl  # #   0&CD C33D  D D))D?>D?zraw_ops.ResourceApplyAdamc8| d} tj| d} | d} tj| d} tj||||||| g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\}}|\}}}}}}} t/j0|tj2}t/j0|tj2}t/j0|tj2}|||||||||| g }d|d| d| f}tj4dd||| | }d}|S)NFrrpr$sResourceApplyAdamrrr)rrrrrorrrrrrrpr#r@rrr:r9r8s r;r$r$sK""; >+L##L.A,66 [RTV[]bdkmq7rtw{B{J{JLSL[L[]d]j]jlslyly{B{H{HJQJVJVX_XiXikrkxkxzAzGzGIPIWIWY`YgYgipiyiy{B{I{IKRKZKZ\c\j\jlsl~l~@G@L@LNUN\N\^e^l^lzop'9@I=; Rw sG$4$45# Q 0 01! Q 0 01!q![+r5%RVW, -n &   11\#)s ?' ' .r=!TV_ResourceApplyAdamWithAmsgrad_Tvhatc Ntjxstj} | j}|jr' t j | d| |||||||||| | d| }|S| d} tj| d} tj d|||||||||| | | | \}}}}|S#t j$r }tj|| Yd}~nd}~wt j$rYnwxYw t|||||||||| | | | | S#t j$rYwxYw)a1Update '*var' according to the Adam algorithm. $$\text{lr}_t := \mathrm{learning_rate} * \sqrt{1 - \beta_2^t} / (1 - \beta_1^t)$$ $$m_t := \beta_1 * m_{t-1} + (1 - \beta_1) * g$$ $$v_t := \beta_2 * v_{t-1} + (1 - \beta_2) * g * g$$ $$\hat{v}_t := max{\hat{v}_{t-1}, v_t}$$ $$\text{variable} := \text{variable} - \text{lr}_t * m_t / (\sqrt{\hat{v}_t} + \epsilon)$$ Args: var: A `Tensor` of type `resource`. Should be from a Variable(). m: A `Tensor` of type `resource`. Should be from a Variable(). v: A `Tensor` of type `resource`. Should be from a Variable(). vhat: A `Tensor` of type `resource`. Should be from a Variable(). beta1_power: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Must be a scalar. beta2_power: A `Tensor`. Must have the same type as `beta1_power`. Must be a scalar. lr: A `Tensor`. Must have the same type as `beta1_power`. Scaling factor. Must be a scalar. beta1: A `Tensor`. Must have the same type as `beta1_power`. Momentum factor. Must be a scalar. beta2: A `Tensor`. Must have the same type as `beta1_power`. Momentum factor. Must be a scalar. epsilon: A `Tensor`. Must have the same type as `beta1_power`. Ridge term. Must be a scalar. grad: A `Tensor`. Must have the same type as `beta1_power`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var, m, and v tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: The created Operation. ResourceApplyAdamWithAmsgradrNrF) rrrr)rrorrrrrrr#)r&rr'r(rrrrrrr/resource_apply_adam_with_amsgrad_eager_fallbackrr*r+r,r-)rrrr)rrorrrrrrr#r3r4r8rr5r6r7s r; resource_apply_adam_with_amsgradr-sZH    0h..0$ #\\  11 ,dCAt["eUGT{$gnK""; >+'88&C14?4?B.3507d4?d L!QX *+  & &- ##At,,  # #    < q!T; R 4[tGG  # #   s0%B**C1=CC10C15DD$#D$z$raw_ops.ResourceApplyAdamWithAmsgradcJ| d} tj| d} tj|||||| | g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\}}|\}}}}}} } t/j0|tj2}t/j0|tj2}t/j0|tj2}t/j0|tj2}|||||||||| | g }d|d| f}tj4dd||| | }d}|S)NFrr$sResourceApplyAdamWithAmsgradrrr)rrrr)rrorrrrrrr#r@rrr:r9r8s r;r,r,8sK""; >+66 [RTV[]bdkmq7rtw{B{J{JLSL[L[]d]j]jlslyly{B{H{HJQJVJVX_XiXikrkxkxzAzGzGIPIWIWY`YgYgipiyiy{B{I{IKRKZKZ\c\j\jlsl~l~@G@L@LNUN\N\^e^l^lzop'9@I=; Rw sG$4$45# Q 0 01! Q 0 01!  g&6&6 7$q!T; RwX\], - 5&   +'88C15+5Dt,7dD!QX *%  & &- ##At,,  # #    3 q"eZt  # #   r zraw_ops.ResourceApplyAddSignc |d}tj|d}tj|||||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}}}t/j0|tj2}t/j0|tj2}|||||||g} d| d|f} tj4dd| | | |}d}|S)NFrr$sResourceApplyAddSignrrr)rrrrzr{r|rrr#r@rrr:r9r8s r;r2r2sVK""; >+66E:tUY7Z\_bibqbqsztCtCELERERT[TaTacjcpcpryr~r~@G@Q@QSZS`S`biboboqxqqAHAOAOQXQaQacjcqcqszsBsBDKDRDRT[TfTfhohthtv}vDvDFMFTFTbWX'9(1%2uj$ sG$4$45# Q 0 01!q"eZt<, - 5&   4a #)s ?' ' .r=!TV_ResourceApplyCenteredRMSProp_Tc Btjxstj} | j} | jr% t j | d| |||||||||d| } | S| d} tj| d} tj d|||||||||| |  \}}}}|S#t j$r }tj|| Yd}~nd}~wt j$rYnwxYw t|||||||||| | |  S#t j$rYwxYw)aUpdate '*var' according to the centered RMSProp algorithm. The centered RMSProp algorithm uses an estimate of the centered second moment (i.e., the variance) for normalization, as opposed to regular RMSProp, which uses the (uncentered) second moment. This often helps with training, but is slightly more expensive in terms of computation and memory. Note that in dense implementation of this algorithm, mg, ms, and mom will update even if the grad is zero, but in this sparse implementation, mg, ms, and mom will not update in iterations during which the grad is zero. mean_square = decay * mean_square + (1-decay) * gradient ** 2 mean_grad = decay * mean_grad + (1-decay) * gradient Delta = learning_rate * gradient / sqrt(mean_square + epsilon - mean_grad ** 2) mg <- rho * mg_{t-1} + (1-rho) * grad ms <- rho * ms_{t-1} + (1-rho) * grad * grad mom <- momentum * mom_{t-1} + lr * grad / sqrt(ms - mg * mg + epsilon) var <- var - mom Args: var: A `Tensor` of type `resource`. Should be from a Variable(). mg: A `Tensor` of type `resource`. Should be from a Variable(). ms: A `Tensor` of type `resource`. Should be from a Variable(). mom: A `Tensor` of type `resource`. Should be from a Variable(). lr: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Scaling factor. Must be a scalar. rho: A `Tensor`. Must have the same type as `lr`. Decay rate. Must be a scalar. momentum: A `Tensor`. Must have the same type as `lr`. Momentum Scale. Must be a scalar. epsilon: A `Tensor`. Must have the same type as `lr`. Ridge term. Must be a scalar. grad: A `Tensor`. Must have the same type as `lr`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var, mg, ms, and mom tensors is protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: The created Operation. ResourceApplyCenteredRMSProprNrFr)r&rr'r(rrrrrrr/resource_apply_centered_rms_prop_eager_fallbackrr*r+r,r-)rrrrrrFrrrrr#r3r4r8rr5r6r7s r; resource_apply_centered_rms_propr9sIZ    0h..0$ #\\ 11 ,dCRb#'4 =gnK""; >+'88&CB232,/(07d4?d L!QX *'  & &- ##At,,  # #    < r2sBXw!$88  # #   rz$raw_ops.ResourceApplyCenteredRMSPropc >| d} tj| d} tj|||||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}}}t/j0|tj2}t/j0|tj2}t/j0|tj2}t/j0|tj2}|||||||||g }d| d| f}tj4dd||| | }d}|S)NFrr$sResourceApplyCenteredRMSProprrr)rrrrrrFrrrrr#r@rrr:r9r8s r;r8r8sK""; >+66C7TX7Y[^ahapaprysBsBDKDQDQSZS`S`biboboqxq}q}FPPRYR_R_ahananpwp~p~@G@N@NPWP`P`bibpbpryrArACJCQCQSZSeSegngsgsu|uCuCELESESaVW'9'0$2sHgt sG$4$45# b'"2"23" b'"2"23" sG$4$45#r2sBXwE, - 5&    l1 else 0.0 accum = accum_new Args: var: A `Tensor` of type `resource`. Should be from a Variable(). accum: A `Tensor` of type `resource`. Should be from a Variable(). linear: A `Tensor` of type `resource`. Should be from a Variable(). grad: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. The gradient. lr: A `Tensor`. Must have the same type as `grad`. Scaling factor. Must be a scalar. l1: A `Tensor`. Must have the same type as `grad`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `grad`. L2 regularization. Must be a scalar. lr_power: A `Tensor`. Must have the same type as `grad`. Scaling factor. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. multiply_linear_by_lr: An optional `bool`. Defaults to `False`. name: A name for the operation (optional). Returns: The created Operation. ResourceApplyFtrlrrNrrr#r@Fr)r&rr'r(rrrrrrr"resource_apply_ftrl_eager_fallbackrr*r+r,r-)rrDrrrr[r\rrrr#r3r4r8rr5r6r7s r;resource_apply_ftrlr@sqB    0h..0$ #\\  11 !4eVT2r2-.EgnK""; >+"!",,-BD[\'88E&t "rb8)43H"& (!QX *1  & &- ##At,,  # #    / ufdBB! 5DdLL  # #   s0$CDC//DD D##D98D9zraw_ops.ResourceApplyFtrlc ,|d}tj|d}| d} tj| d} tj|||||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}}}t/j0|tj2}t/j0|tj2}t/j0|tj2}||||||||g}d| d|d| f}tj4dd||| | }d}|S)NFrrr$sResourceApplyFtrlrrr)rrDrrrr[r\rrrr#r@rrr:r9r8s r;r?r?=sK""; >+"!",,-BD[\66b"b(7SUX[b[j[jlsl{l{~E~K~KMTMZMZ\c\i\ikrkwkwy@yJyJLSLYLY[b[h[hjqjxjxzAzHzHJQJZJZ\c\j\jlsl{l{}D}K}KMTM_M_ahamamovo}o}FMM[PQ'9!*4RX sG$4$45#  (8(8 9%  ! !&'*:*: ;&ufdBBA, -6M &   11\#)s ?' ' .r=TV_ResourceApplyFtrlV2_Tc ~tjxstj} | j} | jr' t j | d| |||||||||d| d| }|S| d} tj| d} | d} tj| d} tj d|||||||||| | |  \}}}}|S#t j$r }tj|| Yd}~nd}~wt j$rYnwxYw t|||||||||| | | |  S#t j$rYwxYw)aUpdate '*var' according to the Ftrl-proximal scheme. accum_new = accum + grad * grad grad_with_shrinkage = grad + 2 * l2_shrinkage * var linear += grad_with_shrinkage + (accum_new^(-lr_power) - accum^(-lr_power)) / lr * var quadratic = 1.0 / (accum_new^(lr_power) * lr) + 2 * l2 var = (sign(linear) * l1 - linear) / quadratic if |linear| > l1 else 0.0 accum = accum_new Args: var: A `Tensor` of type `resource`. Should be from a Variable(). accum: A `Tensor` of type `resource`. Should be from a Variable(). linear: A `Tensor` of type `resource`. Should be from a Variable(). grad: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. The gradient. lr: A `Tensor`. Must have the same type as `grad`. Scaling factor. Must be a scalar. l1: A `Tensor`. Must have the same type as `grad`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `grad`. L2 shrinkage regularization. Must be a scalar. l2_shrinkage: A `Tensor`. Must have the same type as `grad`. lr_power: A `Tensor`. Must have the same type as `grad`. Scaling factor. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. multiply_linear_by_lr: An optional `bool`. Defaults to `False`. name: A name for the operation (optional). Returns: The created Operation. ResourceApplyFtrlV2rrNr>Fr)r&rr'r(rrrrrrr%resource_apply_ftrl_v2_eager_fallbackrr*r+r,r-)rrDrrrr[r\rrrrr#r3r4r8rr5r6r7s r;resource_apply_ftrl_v2rFTswH    0h..0$ #\\  11 #T3vtR L(M;!68gnK""; >+"!",,-BD[\'883eF"$(0k5J$( *!QX *1  & &- ##At,,  # #    2 ufdBB h! 5DdLL  # #   0%CD C11D  D D&&D<;D<zraw_ops.ResourceApplyFtrlV2c 2| d} tj| d} | d} tj| d} tj||||||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} }|\}}}}}}t/j0|tj2}t/j0|tj2}t/j0|tj2}|||||||||g }d| d| d| f}tj4dd||| | }d}|S)NFrrr$sResourceApplyFtrlV2rrr)rrDrrrr[r\rrrrr#r@rrr:r9r8s r;rErEsK""; >+"!",,-BD[\66b"b,X`7acfipixix{B{J{JLSLYLY[b[h[hjqjwjwy@yEyEGNGXGXZaZgZgipivivxxFxFHOHVHVX_XhXhjqjxjxzAzIzIKRKYKY[b[m[movo{o{}D}K}KMTM[M[i^_'9/8,4R\8 sG$4$45#  (8(8 9%  ! !&'*:*: ;&ufdBB hO, -6M &   3Q|#)s ?' ' .r=!TV_ResourceApplyGradientDescent_Tc tjxstj}|j}|jr t j |d||||d|}|S|d}tj|d}tj d|||||\} } } } | S#t j$r }tj||Yd}~nd}~wt j$rYnwxYw t||||||S#t j$rYwxYw)a Update '*var' by subtracting 'alpha' * 'delta' from it. Args: var: A `Tensor` of type `resource`. Should be from a Variable(). alpha: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Scaling factor. Must be a scalar. delta: A `Tensor`. Must have the same type as `alpha`. The change. use_locking: An optional `bool`. Defaults to `False`. If `True`, the subtraction will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: The created Operation. ResourceApplyGradientDescentrNrFr)r&rr'r(rrrrrrr.resource_apply_gradient_descent_eager_fallbackrr*r+r,r-) rrzrrr#r3r4r8rr5r6r7s r;resource_apply_gradient_descentrMs!    0h..0$ #\\ 11 ,dC{$gnK""; >+'88&CuE4?dL!QX *!  & &- ##At,,  # #    ; ue4TKK  # #   s0BC!-CC! C!%C66D  D z$raw_ops.ResourceApplyGradientDescentcN|d}tj|d}tj||g|tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\}}|\}}t/j0|tj2}|||g}d|d|f} tj4dd|| ||} d} | S)NFrr$sResourceApplyGradientDescentrrr) rrzrrr#r@rrr:r9r8s r;rLrLsK""; >+66u~sW__^e^m^movo|o|FLLNUN[N[]d]i]ikrk|k|~E~K~KMTMZMZ\c\j\jlslzlz|C|L|LNUN\N\^e^m^movo}o}FQQSZS_S_ahaoaoqxqqMBC'9.5% sG$4$45#ue$, - 5&   +L##L.A,'88$#Ur/72=3?d L!QX *-  & &- ##At,,  # #    9 ub$k#$D::  # #   rz"raw_ops.ResourceApplyKerasMomentumc |d}tj|d}|d}tj|d}tj|||g|tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}t/j0|tj2}t/j0|tj2}|||||g} d| d|d|f} tj4dd| | ||} d} | S)NFrrpr$sResourceApplyKerasMomentumrrrrrDrrrrrpr#r@rrr:r9r8s r;rRrR1scK""; >+L##L.A,66D(7KSSZSbSbdkdsdsu|vCvCELERERT[TaTacjcocoqxqBqBDKDQDQSZS`S`bibpbpryr@r@BIBRBRT[TbTbdkdsdsu|uCuCELEWEWY`YeYegnguguw~wEwESHI'9"2tX sG$4$45#  (8(8 9%ub$1, -n &   :A$0C"& (' ' .r=TV_ResourceApplyMomentum_Tcftjxstj}|j} | jr# t j |d||||||d|d| } | S|d}tj|d}|d}tj|d}tj d|||||||| \} } } }| S#t j$r } tj| |Yd} ~ nd} ~ wt j$rYnwxYw t||||||||| S#t j$rYwxYw)aUpdate '*var' according to the momentum scheme. Set use_nesterov = True if you want to use Nesterov momentum. accum = accum * momentum + grad var -= lr * accum Args: var: A `Tensor` of type `resource`. Should be from a Variable(). accum: A `Tensor` of type `resource`. Should be from a Variable(). lr: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Scaling factor. Must be a scalar. grad: A `Tensor`. Must have the same type as `lr`. The gradient. momentum: A `Tensor`. Must have the same type as `lr`. Momentum. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. use_nesterov: An optional `bool`. Defaults to `False`. If `True`, the tensor passed to compute grad will be var - lr * momentum * accum, so in the end, the var you get is actually var - lr * momentum * accum. name: A name for the operation (optional). Returns: The created Operation. ResourceApplyMomentumrrpNr#Fr)r&rr'r(rrrrrrr&resource_apply_momentum_eager_fallbackrr*r+r,r-rSs r;resource_apply_momentumr[HsU:    0h..0$ #\\ 11 %tS%T8{NLBgnK""; >+L##L.A,'88S"4*2 .:G!QX *+  & &- ##At,,  # #    3 ub$k#$D::  # #   rzraw_ops.ResourceApplyMomentumc |d}tj|d}|d}tj|d}tj|||g|tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}t/j0|tj2}t/j0|tj2}|||||g} d| d|d|f} tj4dd| | ||} d} | S)NFrrpr$sResourceApplyMomentumrrrrVs r;rZrZs`K""; >+L##L.A,66D(7KSSZSbSbdkdsdsu|vCvCELERERT[TaTacjcocoqxqBqBDKDQDQSZS`S`bibpbpryr@r@BIBRBRT[TbTbdkdsdsu|uCuCELEWEWY`YeYegnguguw~wEwESHI'9"2tX sG$4$45#  (8(8 9%ub$1, -n &   5q#)s ?' ' .r=TV_ResourceApplyPowerSign_Tc 6tjxstj} | j} | jr# t j | d||||||||d| } | S|d}tj|d}tj d||||||||| \} } }}|S#t j$r } tj| |Yd} ~ nd} ~ wt j$rYnwxYw t||||||||||  S#t j$rYwxYw)aUpdate '*var' according to the AddSign update. m_t <- beta1 * m_{t-1} + (1 - beta1) * g update <- exp(logbase * sign_decay * sign(g) * sign(m_t)) * g variable <- variable - lr_t * update Args: var: A `Tensor` of type `resource`. Should be from a Variable(). m: A `Tensor` of type `resource`. Should be from a Variable(). lr: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Scaling factor. Must be a scalar. logbase: A `Tensor`. Must have the same type as `lr`. Must be a scalar. sign_decay: A `Tensor`. Must have the same type as `lr`. Must be a scalar. beta: A `Tensor`. Must have the same type as `lr`. Must be a scalar. grad: A `Tensor`. Must have the same type as `lr`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and m tensors is protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: The created Operation. ResourceApplyPowerSignrNrFr)r&rr'r(rrrrrrr(resource_apply_power_sign_eager_fallbackrr*r+r,r-)rrrrr{r|rrr#r3r4r8rr5r6r7s r;resource_apply_power_signras;2    0h..0$ #\\ 11 &c1b': dM;0gnK""; >+'88 cQ2w-7d.9F!QX *%  & &- ##At,,  # #    5 q"gz4!$88  # #   r zraw_ops.ResourceApplyPowerSignc |d}tj|d}tj|||||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}}}t/j0|tj2}t/j0|tj2}|||||||g} d| d|f} tj4dd| | | |}d}|S)NFrr$sResourceApplyPowerSignrrr)rrrrr{r|rrr#r@rrr:r9r8s r;r`r`sZK""; >+66GZQUW[7\^adkdsdsu|vEvEGNGTGTV]VcVcelerert{t@t@BIBSBSU\UbUbdkdqdqszsAsACJCQCQSZScScelesesu|uDuDFMFTFTV]VhVhjqjvjvxxFxFHOHVHVdYZ'9*3'2w D$ sG$4$45# Q 0 01!q"gz4>, - 5&   6$0C"& (' ' .r=!TV_ResourceApplyProximalAdagrad_Tc 0tjxstj}|j} | jr" t j |d|||||||d| } | S|d}tj|d}tj d|||||||| \} } } }| S#t j$r } tj| |Yd} ~ nd} ~ wt j$rYnwxYw t||||||||| S#t j$rYwxYw)aUpdate '*var' and '*accum' according to FOBOS with Adagrad learning rate. accum += grad * grad prox_v = var - lr * grad * (1 / sqrt(accum)) var = sign(prox_v)/(1+lr*l2) * max{|prox_v|-lr*l1,0} Args: var: A `Tensor` of type `resource`. Should be from a Variable(). accum: A `Tensor` of type `resource`. Should be from a Variable(). lr: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Scaling factor. Must be a scalar. l1: A `Tensor`. Must have the same type as `lr`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `lr`. L2 regularization. Must be a scalar. grad: A `Tensor`. Must have the same type as `lr`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If True, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: The created Operation. ResourceApplyProximalAdagradrNrFr)r&rr'r(rrrrrrr.resource_apply_proximal_adagrad_eager_fallbackrr*r+r,r-)rrDrr[r\rrr#r3r4r8rr5r6r7s r;resource_apply_proximal_adagradrgs52    0h..0$ #\\ 11 ,dCB m[*gnK""; >+'88&Cu+-D4?dL!QX *%  & &- ##At,,  # #    ; ub"b$Kd  # #   0 B C'3CC'&C'+C??DDz$raw_ops.ResourceApplyProximalAdagradc |d}tj|d}tj||||g|tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}}t/j0|tj2}t/j0|tj2}||||||g} d| d|f} tj4dd| | ||} d} | S)NFrr$sResourceApplyProximalAdagradrrr)rrDrr[r\rrr#r@rrr:r9r8s r;rfrf sEK""; >+66BD7I3QXQ`Q`bibqbqsztAtACJCPCPRYR_R_ahamamovo@o@BIBOBOQXQ^Q^`g`n`npwp~p~@G@P@PRYR`R`bibqbqszsAsACJCUCUW^WcWcelesesu|uCuCQFG'9 2r2t sG$4$45#  (8(8 9%ub"b$/, - 5&   +'88.Cu35U+66r2u7MsU\UdUdfmfufuw~xExEGNGTGTV]VcVceleqeqszsDsDFMFSFSU\UbUbdkdrdrt{tBtBDKDTDTV]VdVdfmfufuw~wEwEGNGYGY[b[g[gipiwiwy@yGyGUJK'9$5"b% sG$4$45#ub"e,, - 5&   Da$0C"& (' ' .r=TV_ResourceApplyRMSProp_Tc <tjxstj} | j} | jr$ t j | d| ||||||||d| } | S|d}tj|d}tj d||||||||||  \}}}}|S#t j$r } tj| | Yd} ~ nd} ~ wt j$rYnwxYw t|||||||||| |  S#t j$rYwxYw)atUpdate '*var' according to the RMSProp algorithm. Note that in dense implementation of this algorithm, ms and mom will update even if the grad is zero, but in this sparse implementation, ms and mom will not update in iterations during which the grad is zero. mean_square = decay * mean_square + (1-decay) * gradient ** 2 Delta = learning_rate * gradient / sqrt(mean_square + epsilon) ms <- rho * ms_{t-1} + (1-rho) * grad * grad mom <- momentum * mom_{t-1} + lr * grad / sqrt(ms + epsilon) var <- var - mom Args: var: A `Tensor` of type `resource`. Should be from a Variable(). ms: A `Tensor` of type `resource`. Should be from a Variable(). mom: A `Tensor` of type `resource`. Should be from a Variable(). lr: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Scaling factor. Must be a scalar. rho: A `Tensor`. Must have the same type as `lr`. Decay rate. Must be a scalar. momentum: A `Tensor`. Must have the same type as `lr`. epsilon: A `Tensor`. Must have the same type as `lr`. Ridge term. Must be a scalar. grad: A `Tensor`. Must have the same type as `lr`. The gradient. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var, ms, and mom tensors is protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: The created Operation. ResourceApplyRMSProprNrFr)r&rr'r(rrrrrrr&resource_apply_rms_prop_eager_fallbackrr*r+r,r-)rrrrrFrrrrr#r3r4r8rr5r6r7s r;resource_apply_rms_proprtx sBF    0h..0$ #\\ 11 $dCS"c8}k3gnK""; >+'88CBCBC)17,7dD!QX *%  & &- ##At,,  # #    3 r3C7D!$88  # #   rzraw_ops.ResourceApplyRMSPropc |d}tj|d}tj|||||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}}}t/j0|tj2}t/j0|tj2}t/j0|tj2}||||||||g} d| d|f}tj4dd| || | }d}|S)NFrr$sResourceApplyRMSProprrr)rrrrrFrrrrr#r@rrr:r9r8s r;rsrs spK""; >+66C7TX7Y[^ahapaprysBsBDKDQDQSZS`S`biboboqxq}q}FPPRYR_R_ahananpwp~p~@G@N@NPWP`P`bibpbpryrArACJCQCQSZSeSegngsgsu|uCuCELESESaVW'9'0$2sHgt sG$4$45# b'"2"23" sG$4$45#r3C7DA, - 5&   4a #)s ?' ' .r= TV_ResourceSparseApplyAdadelta_T'TV_ResourceSparseApplyAdadelta_Tindicesindicesc <tjxstj} | j} | jr$ t j | d| ||||||||d| } | S|d}tj|d}tj d||||||||||  \}}}}|S#t j$r } tj| | Yd} ~ nd} ~ wt j$rYnwxYw t|||||||||| |  S#t j$rYwxYw)avar: Should be from a Variable(). Args: var: A `Tensor` of type `resource`. accum: A `Tensor` of type `resource`. Should be from a Variable(). accum_update: A `Tensor` of type `resource`. : Should be from a Variable(). lr: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Learning rate. Must be a scalar. rho: A `Tensor`. Must have the same type as `lr`. Decay factor. Must be a scalar. epsilon: A `Tensor`. Must have the same type as `lr`. Constant factor. Must be a scalar. grad: A `Tensor`. Must have the same type as `lr`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. use_locking: An optional `bool`. Defaults to `False`. If True, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: The created Operation. ResourceSparseApplyAdadeltarNrF rrDrErrFrrrxrr#)r&rr'r(rrrrrrr-resource_sparse_apply_adadelta_eager_fallbackrr*r+r,r-)rrDrErrFrrrxrr#r3r4r8rr5r6r7s r;resource_sparse_apply_adadeltar} sD2    0h..0$ #\\ 11 +T3| C$ EgnK""; >+'88%3e4@R+.d/63>T K!QX *)  & &- ##At,,  # #    : ulBWdG!$88  # #   rz#raw_ops.ResourceSparseApplyAdadeltac j|d}tj|d}tj||||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}}tj|g| tj tjg\} \}t/j0|tj2}t/j0|tj2}t/j0|tj2}||||||||g}d| d| d|f}tj4dd||| | }d}|S)NFrr$TindicessResourceSparseApplyAdadeltarrr)rrDrErrFrrrxrr#r@rr_attr_Tindicesr:r9r8s r;r|r| sK""; >+66C$7OQTW^WfWfhohwhwzAzGzGIPIVIVX_XeXegngsgsu|uFuFHOHUHUW^WdWdfmftftv}vDvDFMFVFVX_XfXfhohwhwy@yGyGIPI[I[]d]i]ikrkyky{B{I{IWLM'9&2sGT'>>y#PWP]P]_f_l_lOop.*7 sG$4$45#  (8(8 9%'' g6F6FG,ulBWdGL, *nm  &   ;Q$0C"& (' ' .r=TV_ResourceSparseApplyAdagrad_T&TV_ResourceSparseApplyAdagrad_Tindicescftjxstj}|j} | jr# t j |d||||||d|d| } | S|d}tj|d}|d}tj|d}tj d|||||||| \} } } }| S#t j$r } tj| |Yd} ~ nd} ~ wt j$rYnwxYw t||||||||| S#t j$rYwxYw) aUpdate relevant entries in '*var' and '*accum' according to the adagrad scheme. That is for rows we have grad for, we update var and accum as follows: accum += grad * grad var -= lr * grad * (1 / sqrt(accum)) Args: var: A `Tensor` of type `resource`. Should be from a Variable(). accum: A `Tensor` of type `resource`. Should be from a Variable(). lr: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Learning rate. Must be a scalar. grad: A `Tensor`. Must have the same type as `lr`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. update_slots: An optional `bool`. Defaults to `True`. name: A name for the operation (optional). Returns: The created Operation. ResourceSparseApplyAdagradrrPNrFTrrDrrrxrrPr#)r&rr'r(rrrrrrr,resource_sparse_apply_adagrad_eager_fallbackrr*r+r,r-)rrDrrrxrrPr#r3r4r8rr5r6r7s r;resource_sparse_apply_adagradr sV2    0h..0$ #\\ 11 *D#ub$ ^\KgnK""; >+L##L.A,'88$#Ur.52=3?d L!QX *-  & &- ##At,,  # #    9 ub$[#$D::  # #   rz"raw_ops.ResourceSparseApplyAdagradc L|d}tj|d}|d}tj|d}tj||g|tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}tj|g|tj tjg\} \}t/j0|tj2}t/j0|tj2}|||||g} d| d| d|d|f} tj4dd| | || }d}|S) NFrTrPr$rsResourceSparseApplyAdagradrrr)rrDrrrxrrPr#r@rrrr:r9r8s r;rrX sK""; >+L##L.A,66Dz3ZaZiZikrkxkx{B{H{HJQJWJWY`YeYegngxgxzAzGzGIPIVIVX_XfXfhohvhvxxHxHJQJXJXZaZiZikrkyky{B{M{MOVO[O[]d]k]kmtm{m{I~'9*2t'>>y#PWP]P]_f_l_lOop.*7 sG$4$45#  (8(8 9%ub$0, *nm ~| -&   :A$0C"& (' ' .r=!TV_ResourceSparseApplyAdagradDA_T(TV_ResourceSparseApplyAdagradDA_Tindicesc Btjxstj} | j} | jr% t j | d| |||||||||d| } | S| d} tj| d} tj d|||||||||| |  \}}}}|S#t j$r }tj|| Yd}~nd}~wt j$rYnwxYw t|||||||||| | |  S#t j$rYwxYw)aUpdate entries in '*var' and '*accum' according to the proximal adagrad scheme. Args: var: A `Tensor` of type `resource`. Should be from a Variable(). gradient_accumulator: A `Tensor` of type `resource`. Should be from a Variable(). gradient_squared_accumulator: A `Tensor` of type `resource`. Should be from a Variable(). grad: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. lr: A `Tensor`. Must have the same type as `grad`. Learning rate. Must be a scalar. l1: A `Tensor`. Must have the same type as `grad`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `grad`. L2 regularization. Must be a scalar. global_step: A `Tensor` of type `int64`. Training step number. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If True, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: The created Operation. ResourceSparseApplyAdagradDArNrF rrYrZrrxrr[r\r]rr#)r&rr'r(rrrrrrr/resource_sparse_apply_adagrad_da_eager_fallbackrr*r+r,r-)rrYrZrrxrr[r\r]rr#r3r4r8rr5r6r7s r; resource_sparse_apply_adagrad_darq sO:    0h..0$ #\\  11 ,dC9M$dGRR{$gnK""; >+'88&C=QEa-17r+-"+4?d L!QX *-  & &- ##At,,  # #    < #%A4 2r2{   # #   rz$raw_ops.ResourceSparseApplyAdagradDAc | d} tj| d} tj||||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}}tj|g| tj tjg\}\}t/j0|tj2}t/j0|tj2}t/j0|tj2}t/j0|tj}|||||||||g }d| d|d| f}tj4dd||| | }d}|S)NFrr$rsResourceSparseApplyAdagradDArrr)rrYrZrrxrr[r\r]rr#r@rrrr:r9r8s r;rr sK""; >+66b"b7I3QXQ`Q`bibqbqsztAtACJCPCPRYR_R_ahamamovo@o@BIBOBOQXQ^Q^`g`n`npwp~p~@G@P@PRYR`R`bibqbqszsAsACJCUCUW^WcWcelesesu|uCuCQFG'9 4R'>>y#PWP]P]_f_l_lOop.*7 sG$4$45#//0DgFVFVW!%!7!78TV]VfVf!g&&{GMMB++-I4QXZ\^`bdfqr, *nm  &   gnK""; >+L##L.A,'88&Cu07d074?5A N!QX */  & &- ##At,,  # #    < ub'4k#$D::  # #   0"B==DC++DDDD32D3z$raw_ops.ResourceSparseApplyAdagradV2c R|d}tj|d}|d}tj|d}tj|||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}tj|g| tj tjg\} \}t/j0|tj2}t/j0|tj2}||||||g} d| d| d|d|f}tj4dd| || | }d}|S) NFrTrPr$rsResourceSparseApplyAdagradV2rrr)rrDrrrrxrrPr#r@rrrr:r9r8s r;rr sK""; >+L##L.A,66GT7JCRYRaRacjcrcrt{uBuBDKDQDQSZS`S`bibnbnpwpApACJCPCPRYR_R_ahaoaoqxqqAHAQAQSZSaSacjcrcrt{tBtBDKDVDVX_XdXdfmftftv}vDvDRGH'9!2w'>>y#PWP]P]_f_l_lOop.*7 sG$4$45#  (8(8 9%ub'49, *nm ~| -&   +'88,#"137?6=D6=:E379!QX *-  & &- ##At,,  # #    C r2sBXwg!$88  # #   s0$B((C/;CC/.C/3D D! D!z*raw_ops.ResourceSparseApplyCenteredRMSPropc | d} tj| d} tj|||||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} }|\}}}}}tj| g| tj tjg\}\} t/j0|tj2}t/j0|tj2}t/j0|tj2}t/j0|tj2}|||||||||| g }d| d|d| f}tj4dd||| | }d}|S)NFrr$rs"ResourceSparseApplyCenteredRMSProprrr)rrrrrrFrrrrxrr#r@rrrr:r9r8s r;rro sK""; >+66C7TX7Y[^ahapaprysBsBDKDQDQSZS`S`biboboqxq}q}FPPRYR_R_ahananpwp~p~@G@N@NPWP`P`bibpbpryrArACJCQCQSZSeSegngsgsu|uCuCELESESaVW'9'0$2sHgt'>>y#PWP]P]_f_l_lOop.*7 sG$4$45# b'"2"23" b'"2"23" sG$4$45#r2sBXwgN, *nm  &   BA$0C"& (' ' .r=TV_ResourceSparseApplyFtrl_T#TV_ResourceSparseApplyFtrl_Tindicesc ~tjxstj} | j} | jr' t j | d| |||||||||d| d| }|S| d} tj| d} | d} tj| d} tj d|||||||||| | |  \}}}}|S#t j$r }tj|| Yd}~nd}~wt j$rYnwxYw t|||||||||| | | |  S#t j$rYwxYw)a#Update relevant entries in '*var' according to the Ftrl-proximal scheme. That is for rows we have grad for, we update var, accum and linear as follows: accum_new = accum + grad * grad linear += grad - (accum_new^(-lr_power) - accum^(-lr_power)) / lr * var quadratic = 1.0 / (accum_new^(lr_power) * lr) + 2 * l2 var = (sign(linear) * l1 - linear) / quadratic if |linear| > l1 else 0.0 accum = accum_new Args: var: A `Tensor` of type `resource`. Should be from a Variable(). accum: A `Tensor` of type `resource`. Should be from a Variable(). linear: A `Tensor` of type `resource`. Should be from a Variable(). grad: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. lr: A `Tensor`. Must have the same type as `grad`. Scaling factor. Must be a scalar. l1: A `Tensor`. Must have the same type as `grad`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `grad`. L2 regularization. Must be a scalar. lr_power: A `Tensor`. Must have the same type as `grad`. Scaling factor. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. multiply_linear_by_lr: An optional `bool`. Defaults to `False`. name: A name for the operation (optional). Returns: The created Operation. ResourceSparseApplyFtrlrrNr>F rrDrrrxrr[r\rrrr#)r&rr'r(rrrrrrr)resource_sparse_apply_ftrl_eager_fallbackrr*r+r,r-)rrDrrrxrr[r\rrrr#r3r4r8rr5r6r7s r;resource_sparse_apply_ftrlr sxH    0h..0$ #\\  11 'sE64RX}k!68gnK""; >+"!",,-BD[\'88!s%(,g"&(8/:9N(, .!QX *3  & &- ##At,,  # #    6 ufdGRR! 5DdLL  # #   rGzraw_ops.ResourceSparseApplyFtrlc | d} tj| d} | d} tj| d} tj|||||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} }|\}}}}}tj|g| tj tjg\}\}t/j0|tj2}t/j0|tj2}t/j0|tj2}|||||||||g }d| d|d| d| f}tj4dd||| | }d}|S) NFrrr$rsResourceSparseApplyFtrlrrr)rrDrrrxrr[r\rrrr#r@rrrr:r9r8s r;rr sK""; >+"!",,-BD[\66b"b(7SUX[b[j[jlsl{l{~E~K~KMTMZMZ\c\i\ikrkwkwy@yJyJLSLYLY[b[h[hjqjxjxzAzHzHJQJZJZ\c\j\jlsl{l{}D}K}KMTM_M_ahamamovo}o}FMM[PQ'9!*4RX'>>y#PWP]P]_f_l_lOop.*7 sG$4$45#  (8(8 9%  ! !&'*:*: ;&ufdGRRJ, *nm &(= ?&   7$0C"& (' ' .r=TV_ResourceSparseApplyFtrlV2_T%TV_ResourceSparseApplyFtrlV2_Tindicesc tjxstj} | j}|jr( t j | d| |||||||||| d| d| }|S| d} tj| d} | d} tj| d} tj d|||||||||| | | | \}}}}|S#t j$r }tj|| Yd}~nd}~wt j$rYnwxYw t|||||||||| | | | | S#t j$rYwxYw)aUpdate relevant entries in '*var' according to the Ftrl-proximal scheme. That is for rows we have grad for, we update var, accum and linear as follows: grad_with_shrinkage = grad + 2 * l2_shrinkage * var accum_new = accum + grad_with_shrinkage * grad_with_shrinkage linear += grad_with_shrinkage + (accum_new^(-lr_power) - accum^(-lr_power)) / lr * var quadratic = 1.0 / (accum_new^(lr_power) * lr) + 2 * l2 var = (sign(linear) * l1 - linear) / quadratic if |linear| > l1 else 0.0 accum = accum_new Args: var: A `Tensor` of type `resource`. Should be from a Variable(). accum: A `Tensor` of type `resource`. Should be from a Variable(). linear: A `Tensor` of type `resource`. Should be from a Variable(). grad: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. lr: A `Tensor`. Must have the same type as `grad`. Scaling factor. Must be a scalar. l1: A `Tensor`. Must have the same type as `grad`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `grad`. L2 shrinkage regularization. Must be a scalar. l2_shrinkage: A `Tensor`. Must have the same type as `grad`. lr_power: A `Tensor`. Must have the same type as `grad`. Scaling factor. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. multiply_linear_by_lr: An optional `bool`. Defaults to `False`. name: A name for the operation (optional). Returns: The created Operation. ResourceSparseApplyFtrlV2rrNr>F rrDrrrxrr[r\rrrrr#)r&rr'r(rrrrrrr,resource_sparse_apply_ftrl_v2_eager_fallbackrr*r+r,r-)rrDrrrxrr[r\rrrrr#r3r4r8rr5r6r7s r;resource_sparse_apply_ftrl_v2r sN    0h..0$ #\\  11 )4eVTR\8],.CEgnK""; >+"!",,-BD[\'88#E&*.B2(*.61<;P*.0!QX *5  & &- ##At,,  # #    9 ufdGRR  5DdLL  # #   r&z!raw_ops.ResourceSparseApplyFtrlV2c| d} tj| d} | d} tj| d} tj|||||| g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\}}|\}}}}}} tj|g| tj tjg\}\}t/j0|tj2}t/j0|tj2}t/j0|tj2}|||||||||| g }d|d|d| d| f}tj4dd||| | }d}|S) NFrrr$rsResourceSparseApplyFtrlV2rrr)rrDrrrxrr[r\rrrrr#r@rrrr:r9r8s r;rr9 sK""; >+"!",,-BD[\66b"b,X`7acfipixix{B{J{JLSLYLY[b[h[hjqjwjwy@yEyEGNGXGXZaZgZgipivivxxFxFHOHVHVX_XhXhjqjxjxzAzIzIKRKYKY[b[m[movo{o{}D}K}KMTM[M[i^_'9/8,4R\8'>>y#PWP]P]_f_l_lOop.*7 sG$4$45#  (8(8 9%  ! !&'*:*: ;&ufdGRRxX, *nm &(= ?&   91$0C"& (' ' .r=%TV_ResourceSparseApplyKerasMomentum_T,TV_ResourceSparseApplyKerasMomentum_Tindicesc ltjxstj} | j} | jr$ t j | d|||||||d|d| } | S|d}tj|d}|d}tj|d}tj d||||||||| \} } }}|S#t j$r } tj| |Yd} ~ nd} ~ wt j$rYnwxYw t||||||||||  S#t j$rYwxYw)aUpdate relevant entries in '*var' and '*accum' according to the momentum scheme. Set use_nesterov = True if you want to use Nesterov momentum. That is for rows we have grad for, we update var and accum as follows: accum = accum * momentum - lr * grad var += accum Args: var: A `Tensor` of type `resource`. Should be from a Variable(). accum: A `Tensor` of type `resource`. Should be from a Variable(). lr: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Learning rate. Must be a scalar. grad: A `Tensor`. Must have the same type as `lr`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. momentum: A `Tensor`. Must have the same type as `lr`. Momentum. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. use_nesterov: An optional `bool`. Defaults to `False`. If `True`, the tensor passed to compute grad will be var + momentum * accum, so in the end, the var you get is actually var + momentum * accum. name: A name for the operation (optional). Returns: The created Operation. ResourceSparseApplyKerasMomentumrrpNr#F rrDrrrxrrrpr#)r&rr'r(rrrrrrr3resource_sparse_apply_keras_momentum_eager_fallbackrr*r+r,r-rrDrrrxrrrpr#r3r4r8rr5r6r7s r;$resource_sparse_apply_keras_momentumrS s^B    0h..0$ #\\  11 0$UB=+~gnK""; >+L##L.A,'88*5R15w5=8C9E15 7!QX *1  & &- ##At,,  # #    @ ub${#$D::  # #   rz(raw_ops.ResourceSparseApplyKerasMomentumc R|d}tj|d}|d}tj|d}tj|||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}tj|g| tj tjg\} \}t/j0|tj2}t/j0|tj2}||||||g} d| d| d|d|f}tj4dd| || |}d}|S) NFrrpr$rs ResourceSparseApplyKerasMomentumrrrrrDrrrxrrrpr#r@rrrr:r9r8s r;rr sK""; >+L##L.A,66D(7KSSZSbSbdkdsdsu|vCvCELERERT[TaTacjcocoqxqBqBDKDQDQSZS`S`bibpbpryr@r@BIBRBRT[TbTbdkdsdsu|uCuCELEWEWY`YeYegnguguw~wEwESHI'9"2tX'>>y#PWP]P]_f_l_lOop.*7 sG$4$45#  (8(8 9%ub$:, *nm ~| -&   @!$0C"& (' ' .r= TV_ResourceSparseApplyMomentum_T'TV_ResourceSparseApplyMomentum_Tindicesc ltjxstj} | j} | jr$ t j | d|||||||d|d| } | S|d}tj|d}|d}tj|d}tj d||||||||| \} } }}|S#t j$r } tj| |Yd} ~ nd} ~ wt j$rYnwxYw t||||||||||  S#t j$rYwxYw)aUpdate relevant entries in '*var' and '*accum' according to the momentum scheme. Set use_nesterov = True if you want to use Nesterov momentum. That is for rows we have grad for, we update var and accum as follows: accum = accum * momentum + grad var -= lr * accum Args: var: A `Tensor` of type `resource`. Should be from a Variable(). accum: A `Tensor` of type `resource`. Should be from a Variable(). lr: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Learning rate. Must be a scalar. grad: A `Tensor`. Must have the same type as `lr`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. momentum: A `Tensor`. Must have the same type as `lr`. Momentum. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. use_nesterov: An optional `bool`. Defaults to `False`. If `True`, the tensor passed to compute grad will be var - lr * momentum * accum, so in the end, the var you get is actually var - lr * momentum * accum. name: A name for the operation (optional). Returns: The created Operation. ResourceSparseApplyMomentumrrpNr#Fr)r&rr'r(rrrrrrr-resource_sparse_apply_momentum_eager_fallbackrr*r+r,r-rs r;resource_sparse_apply_momentumr s]B    0h..0$ #\\  11 +T3r4=+~gnK""; >+L##L.A,'88%3e/63>4@t M!QX *-  & &- ##At,,  # #    : ub${#$D::  # #   rz#raw_ops.ResourceSparseApplyMomentumc R|d}tj|d}|d}tj|d}tj|||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}tj|g| tj tjg\} \}t/j0|tj2}t/j0|tj2}||||||g} d| d| d|d|f}tj4dd| || |}d}|S) NFrrpr$rsResourceSparseApplyMomentumrrrrs r;rr sK""; >+L##L.A,66D(7KSSZSbSbdkdsdsu|vCvCELERERT[TaTacjcocoqxqBqBDKDQDQSZS`S`bibpbpryr@r@BIBRBRT[TbTbdkdsdsu|uCuCELEWEWY`YeYegnguguw~wEwESHI'9"2tX'>>y#PWP]P]_f_l_lOop.*7 sG$4$45#  (8(8 9%ub$:, *nm ~| -&   ;Q$0C"& (' ' .r='TV_ResourceSparseApplyProximalAdagrad_T.TV_ResourceSparseApplyProximalAdagrad_Tindicesc 6tjxstj} | j} | jr# t j | d||||||||d| } | S|d}tj|d}tj d||||||||| \} } }}|S#t j$r } tj| |Yd} ~ nd} ~ wt j$rYnwxYw t||||||||||  S#t j$rYwxYw)aSparse update entries in '*var' and '*accum' according to FOBOS algorithm. That is for rows we have grad for, we update var and accum as follows: accum += grad * grad prox_v = var prox_v -= lr * grad * (1 / sqrt(accum)) var = sign(prox_v)/(1+lr*l2) * max{|prox_v|-lr*l1,0} Args: var: A `Tensor` of type `resource`. Should be from a Variable(). accum: A `Tensor` of type `resource`. Should be from a Variable(). lr: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Learning rate. Must be a scalar. l1: A `Tensor`. Must have the same type as `lr`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `lr`. L2 regularization. Must be a scalar. grad: A `Tensor`. Must have the same type as `lr`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. use_locking: An optional `bool`. Defaults to `False`. If True, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: The created Operation. "ResourceSparseApplyProximalAdagradrNrF rrDrr[r\rrxrr#)r&rr'r(rrrrrrr5resource_sparse_apply_proximal_adagrad_eager_fallbackrr*r+r,r-)rrDrr[r\rrxrr#r3r4r8rr5r6r7s r;&resource_sparse_apply_proximal_adagradr s<:    0h..0$ #\\ 11 2D#ub" D'=+7gnK""; >+'88,#Ur136=:E37 9!QX *)  & &- ##At,,  # #    B ub"b$[  # #   r z*raw_ops.ResourceSparseApplyProximalAdagradc  |d}tj|d}tj||||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}}tj|g| tj tjg\} \}t/j0|tj2}t/j0|tj2}|||||||g} d| d| d|f}tj4dd| || |}d}|S)NFrr$rs"ResourceSparseApplyProximalAdagradrrr)rrDrr[r\rrxrr#r@rrrr:r9r8s r;rrL s~K""; >+66BD7I3QXQ`Q`bibqbqsztAtACJCPCPRYR_R_ahamamovo@o@BIBOBOQXQ^Q^`g`n`npwp~p~@G@P@PRYR`R`bibqbqszsAsACJCUCUW^WcWcelesesu|uCuCQFG'9 2r2t'>>y#PWP]P]_f_l_lOop.*7 sG$4$45#  (8(8 9%ub"b$8, *nm  &   BA$0C"& (' ' .r=/TV_ResourceSparseApplyProximalGradientDescent_T6TV_ResourceSparseApplyProximalGradientDescent_Tindicesc 0tjxstj}|j} | jr" t j |d|||||||d| } | S|d}tj|d}tj d|||||||| \} } } }| S#t j$r } tj| |Yd} ~ nd} ~ wt j$rYnwxYw t||||||||| S#t j$rYwxYw)a Sparse update '*var' as FOBOS algorithm with fixed learning rate. That is for rows we have grad for, we update var as follows: prox_v = var - alpha * grad var = sign(prox_v)/(1+alpha*l2) * max{|prox_v|-alpha*l1,0} Args: var: A `Tensor` of type `resource`. Should be from a Variable(). alpha: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Scaling factor. Must be a scalar. l1: A `Tensor`. Must have the same type as `alpha`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `alpha`. L2 regularization. Must be a scalar. grad: A `Tensor`. Must have the same type as `alpha`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. use_locking: An optional `bool`. Defaults to `False`. If True, the subtraction will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: The created Operation. *ResourceSparseApplyProximalGradientDescentrNrFrrzr[r\rrxrr#)r&rr'r(rrrrrrr>resource_sparse_apply_proximal_gradient_descent_eager_fallbackrr*r+r,r-)rrzr[r\rrxrr#r3r4r8rr5r6r7s r;/resource_sparse_apply_proximal_gradient_descentrb s74    0h..0$ #\\ 11 :D#u Bg}k;gnK""; >+'884#U9;>EBM;? A!QX *)  & &- ##At,,  # #    K ub"dG  # #   rhz2raw_ops.ResourceSparseApplyProximalGradientDescentc |d}tj|d}tj||||g|tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}}tj|g|tj tjg\} \}t/j0|tj2}||||||g} d| d| d|f} tj4dd| | ||}d}|S)NFrr$rs*ResourceSparseApplyProximalGradientDescentrrr)rrzr[r\rrxrr#r@rrrr:r9r8s r;rr sdK""; >+66r2t7LcT[TcTceletetv}wDwDFMFSFSU\UbUbdkdpdpryrCrCELERERT[TaTacjcqcqszsAsACJCSCSU\UcUceletetv}vDvDFMFXFXZaZfZfhohvhvxxFxFTIJ'9#5"b$'>>y#PWP]P]_f_l_lOop.*7 sG$4$45#ub"dG4, *nm  &   JA$0C"& (' ' .r=TV_ResourceSparseApplyRMSProp_T&TV_ResourceSparseApplyRMSProp_Tindicesc Btjxstj} | j} | jr% t j | d| |||||||||d| } | S| d} tj| d} tj d|||||||||| |  \}}}}|S#t j$r }tj|| Yd}~nd}~wt j$rYnwxYw t|||||||||| | |  S#t j$rYwxYw)a Update '*var' according to the RMSProp algorithm. Note that in dense implementation of this algorithm, ms and mom will update even if the grad is zero, but in this sparse implementation, ms and mom will not update in iterations during which the grad is zero. mean_square = decay * mean_square + (1-decay) * gradient ** 2 Delta = learning_rate * gradient / sqrt(mean_square + epsilon) ms <- rho * ms_{t-1} + (1-rho) * grad * grad mom <- momentum * mom_{t-1} + lr * grad / sqrt(ms + epsilon) var <- var - mom Args: var: A `Tensor` of type `resource`. Should be from a Variable(). ms: A `Tensor` of type `resource`. Should be from a Variable(). mom: A `Tensor` of type `resource`. Should be from a Variable(). lr: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Scaling factor. Must be a scalar. rho: A `Tensor`. Must have the same type as `lr`. Decay rate. Must be a scalar. momentum: A `Tensor`. Must have the same type as `lr`. epsilon: A `Tensor`. Must have the same type as `lr`. Ridge term. Must be a scalar. grad: A `Tensor`. Must have the same type as `lr`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var, ms and mom. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var, ms, and mom tensors is protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: The created Operation. ResourceSparseApplyRMSProprNrF rrrrrFrrrrxrr#)r&rr'r(rrrrrrr-resource_sparse_apply_rms_prop_eager_fallbackrr*r+r,r-)rrrrrFrrrrxrr#r3r4r8rr5r6r7s r;resource_sparse_apply_rms_propr sJJ    0h..0$ #\\ 11 *D#r3C'4-FgnK""; >+'88$#"#"#/7+/2=D J!QX *'  & &- ##At,,  # #    : r3C7D'!$88  # #   rz"raw_ops.ResourceSparseApplyRMSPropc p| d} tj| d} tj|||||g| tjtj tj tjtjtjtjtjtjtjtjtjtj tj"tj$tj&tj(tj*tj,g\} } | \}}}}}tj|g| tj tjg\}\}t/j0|tj2}t/j0|tj2}t/j0|tj2}|||||||||g }d| d|d| f}tj4dd||| | }d}|S)NFrr$rsResourceSparseApplyRMSProprrr)rrrrrFrrrrxrr#r@rrrr:r9r8s r;rr sK""; >+66C7TX7Y[^ahapaprysBsBDKDQDQSZS`S`biboboqxq}q}FPPRYR_R_ahananpwp~p~@G@N@NPWP`P`bibpbpryrArACJCQCQSZSeSegngsgsu|uCuCELESESaVW'9'0$2sHgt'>>y#PWP]P]_f_l_lOop.*7 sG$4$45# b'"2"23" sG$4$45#r3C7D'J, *nm  &   :A$0C"& (' ' .r=TV_SparseApplyAdadelta_TTV_SparseApplyAdadelta_Tindicesc tjxstj} | j} | jr t d|d}t j |d}tjd||||||||||  \} } } }|dd}t jrYd| jdd| jdd| jdf}| j}t jd||||\}|S) a?var: Should be from a Variable(). Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. accum: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). accum_update: A mutable `Tensor`. Must have the same type as `var`. : Should be from a Variable(). lr: A `Tensor`. Must have the same type as `var`. Learning rate. Must be a scalar. rho: A `Tensor`. Must have the same type as `var`. Decay factor. Must be a scalar. epsilon: A `Tensor`. Must have the same type as `var`. Constant factor. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. use_locking: An optional `bool`. Defaults to `False`. If True, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Nsparse_apply_adadelta op does not support eager execution. Arg 'out' is a ref.NFrSparseApplyAdadeltar{r$rr%)rrDrErrFrrrxrr#r3r4r5r6r7r8r9r:s r;sparse_apply_adadeltar s4    0h..0$ #\\ g hhK""; >+'883e,8RS'.T7+6T C!QX QK' ""$3%%c*J  ,m  /1F::L |VW> (' .r=zraw_ops.SparseApplyAdadeltac td)Nrr?) rrDrErrFrrrxrr#r@s r;$sparse_apply_adadelta_eager_fallbackrAeffr=TV_SparseApplyAdagrad_TTV_SparseApplyAdagrad_Tindicesc Ttjxstj}|j} | jr t d|d}t j |d}|d}t j |d}tjd|||||||| \} } } } | dd} t jrjd | jd d | jd d| jdd| jdf}| j}t jd||| | \} | S) a4Update relevant entries in '*var' and '*accum' according to the adagrad scheme. That is for rows we have grad for, we update var and accum as follows: $$accum += grad * grad$$ $$var -= lr * grad * (1 / sqrt(accum))$$ Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). accum: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). lr: A `Tensor`. Must have the same type as `var`. Learning rate. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. update_slots: An optional `bool`. Defaults to `True`. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Msparse_apply_adagrad op does not support eager execution. Arg 'out' is a ref.NFrTrPSparseApplyAdagradrr$rr%)rrDrrrxrrPr#r3r4r5r6r7r8r9r:s r;sparse_apply_adagradrGs06    0h..0$ #\\ f ggK""; >+L##L.A,'88#Ur&-;+7dD!QX QK' ""$3%%c*J  ,m  /  02F::L lFG= (' .r=zraw_ops.SparseApplyAdagradc td)Nrr?) rrDrrrxrrPr#r@s r;#sparse_apply_adagrad_eager_fallbackrdeer=TV_SparseApplyAdagradDA_T TV_SparseApplyAdagradDA_Tindicesc tjxstj} | j} | jr t d| d} t j | d} tjd|||||||||| |  \} } }}|dd}t jrYd|jdd|jdd|jdf}|j}t jd||||\}|S) a Update entries in '*var' and '*accum' according to the proximal adagrad scheme. Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). gradient_accumulator: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). gradient_squared_accumulator: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). grad: A `Tensor`. Must have the same type as `var`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. lr: A `Tensor`. Must have the same type as `var`. Learning rate. Must be a scalar. l1: A `Tensor`. Must have the same type as `var`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `var`. L2 regularization. Must be a scalar. global_step: A `Tensor` of type `int64`. Training step number. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If True, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Psparse_apply_adagrad_da op does not support eager execution. Arg 'out' is a ref.NFrSparseApplyAdagradDArr$rr%)rrYrZrrxrr[r\r]rr#r3r4r5r6r7r8r9r:s r;sparse_apply_adagrad_dars :    0h..0$ #\\ i jjK""; >+'88C5I=Y%)7rb#%;,7d D!QX QK' ""$3%%c*J  ,m  /1F::L  fg? (' .r=zraw_ops.SparseApplyAdagradDAc td)Nrr?) rrYrZrrxrr[r\r]rr#r@s r;&sparse_apply_adagrad_da_eager_fallbackrrr=TV_SparseApplyAdagradV2_T TV_SparseApplyAdagradV2_Tindicesc Vtjxstj} | j} | jr t d|d}t j |d}|d}t j |d}tjd||||||||| \} } } } | dd}t jrjd | jd d | jd d| jdd| jdf}| j}t jd||||\}|S) aUpdate relevant entries in '*var' and '*accum' according to the adagrad scheme. That is for rows we have grad for, we update var and accum as follows: $$accum += grad * grad$$ $$var -= lr * grad * (1 / sqrt(accum))$$ Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). accum: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). lr: A `Tensor`. Must have the same type as `var`. Learning rate. Must be a scalar. epsilon: A `Tensor`. Must have the same type as `var`. Constant factor. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. update_slots: An optional `bool`. Defaults to `True`. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Psparse_apply_adagrad_v2 op does not support eager execution. Arg 'out' is a ref.NFrTrPSparseApplyAdagradV2rr$rr%)rrDrrrrxrrPr#r3r4r5r6r7r8r9r:s r;sparse_apply_adagrad_v2rs3:    0h..0$ #\\ i jjK""; >+L##L.A,'88CuW%)7,7-9 F!QX QK' ""$3%%c*J  ,m  /  02F::L  fg? (' .r=zraw_ops.SparseApplyAdagradV2c td)Nrr?) rrDrrrrxrrPr#r@s r;&sparse_apply_adagrad_v2_eager_fallbackrrr=TV_SparseApplyCenteredRMSProp_T&TV_SparseApplyCenteredRMSProp_Tindicesc tjxstj} | j} | jr t d| d} t j | d} tjd|||||||||| | |  \}}}}|dd}t jrYd|jdd|jdd|jdf}|j}t jd||||\}|S) av Update '*var' according to the centered RMSProp algorithm. The centered RMSProp algorithm uses an estimate of the centered second moment (i.e., the variance) for normalization, as opposed to regular RMSProp, which uses the (uncentered) second moment. This often helps with training, but is slightly more expensive in terms of computation and memory. Note that in dense implementation of this algorithm, mg, ms, and mom will update even if the grad is zero, but in this sparse implementation, mg, ms, and mom will not update in iterations during which the grad is zero. mean_square = decay * mean_square + (1-decay) * gradient ** 2 mean_grad = decay * mean_grad + (1-decay) * gradient Delta = learning_rate * gradient / sqrt(mean_square + epsilon - mean_grad ** 2) $$ms <- rho * ms_{t-1} + (1-rho) * grad * grad$$ $$mom <- momentum * mom_{t-1} + lr * grad / sqrt(ms + epsilon)$$ $$var <- var - mom$$ Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). mg: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). ms: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). mom: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. rho: A `Tensor`. Must have the same type as `var`. Decay rate. Must be a scalar. momentum: A `Tensor`. Must have the same type as `var`. epsilon: A `Tensor`. Must have the same type as `var`. Ridge term. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var, ms and mom. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var, mg, ms, and mom tensors is protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Wsparse_apply_centered_rms_prop op does not support eager execution. Arg 'out' is a ref.NFrSparseApplyCenteredRMSProprr$rr%)rrrrrrFrrrrxrr#r3r4r5r6r7r8r9r:s r;sparse_apply_centered_rms_proprs`    0h..0$ #\\ p qqK""; >+'88$#"*-.5D.52=D J!QX QK' ""$3%%c*J  ,m  /1F::L $lFGE (' .r=z"raw_ops.SparseApplyCenteredRMSPropc td)Nrr?) rrrrrrFrrrrxrr#r@s r;-sparse_apply_centered_rms_prop_eager_fallbackrTsnoor=TV_SparseApplyFtrl_TTV_SparseApplyFtrl_Tindicesc \tjxstj} | j} | jr t d| d} t j | d} | d} t j | d} tjd|||||||||| | |  \}}}}|dd}t jrjd|jdd |jd d|jdd|jdf}|j}t jd||||\}|S) aUpdate relevant entries in '*var' according to the Ftrl-proximal scheme. That is for rows we have grad for, we update var, accum and linear as follows: $$accum_new = accum + grad * grad$$ $$linear += grad + (accum_{new}^{-lr_{power}} - accum^{-lr_{power}} / lr * var$$ $$quadratic = 1.0 / (accum_{new}^{lr_{power}} * lr) + 2 * l2$$ $$var = (sign(linear) * l1 - linear) / quadratic\ if\ |linear| > l1\ else\ 0.0$$ $$accum = accum_{new}$$ Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). accum: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). linear: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). grad: A `Tensor`. Must have the same type as `var`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. l1: A `Tensor`. Must have the same type as `var`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `var`. L2 regularization. Must be a scalar. lr_power: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. multiply_linear_by_lr: An optional `bool`. Defaults to `False`. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Jsparse_apply_ftrl op does not support eager execution. Arg 'out' is a ref.NFrrSparseApplyFtrlrr$rr%)rrDrrrxrr[r\rrrr#r3r4r5r6r7r8r9r:s r;sparse_apply_ftrlr Zs@L    0h..0$ #\\ c ddK""; >+"!",,-BD[\'88s%T#*rbR$,+1F $ &!QX QK' ""$3%%c*J  ,m  /1H  !89;F::L <: (' .r=zraw_ops.SparseApplyFtrlc td)Nrr?) rrDrrrxrr[r\rrrr#r@s r; sparse_apply_ftrl_eager_fallbackr sabbr=TV_SparseApplyFtrlV2_TTV_SparseApplyFtrlV2_Tindicesc ^tjxstj} | j}|jr t d| d} t j | d} | d} t j | d} tjd|||||||||| | | | \}}}}|dd}t jrjd|jdd |jd d|jdd|jdf}|j}t jd||||\}|S) aUpdate relevant entries in '*var' according to the Ftrl-proximal scheme. That is for rows we have grad for, we update var, accum and linear as follows: grad_with_shrinkage = grad + 2 * l2_shrinkage * var accum_new = accum + grad * grad linear += grad_with_shrinkage - (accum_new^(-lr_power) - accum^(-lr_power)) / lr * var quadratic = 1.0 / (accum_new^(lr_power) * lr) + 2 * l2 var = (sign(linear) * l1 - linear) / quadratic if |linear| > l1 else 0.0 accum = accum_new Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). accum: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). linear: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). grad: A `Tensor`. Must have the same type as `var`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. l1: A `Tensor`. Must have the same type as `var`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `var`. L2 shrinkage regularization. Must be a scalar. l2_shrinkage: A `Tensor`. Must have the same type as `var`. lr_power: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. multiply_linear_by_lr: An optional `bool`. Defaults to `False`. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Msparse_apply_ftrl_v2 op does not support eager execution. Arg 'out' is a ref.NFrrSparseApplyFtrlV2rr$rr%)rrDrrrxrr[r\rrrrr#r3r4r5r6r7r8r9r:s r;sparse_apply_ftrl_v2rsCR    0h..0$ #\\ f ggK""; >+"!",,-BD[\'88E&t%,r*6)43H"& (!QX QK' ""$3%%c*J  ,m  /1H  !89;F::L \67< (' .r=zraw_ops.SparseApplyFtrlV2ctd)Nrr?)rrDrrrxrr[r\rrrrr#r@s r;#sparse_apply_ftrl_v2_eager_fallbackrrr=TV_SparseApplyMomentum_TTV_SparseApplyMomentum_Tindicesc Vtjxstj} | j} | jr t d|d}t j |d}|d}t j |d}tjd||||||||| \} } } } | dd}t jrjd| jdd | jd d| jdd| jdf}| j}t jd||||\}|S) azUpdate relevant entries in '*var' and '*accum' according to the momentum scheme. Set use_nesterov = True if you want to use Nesterov momentum. That is for rows we have grad for, we update var and accum as follows: $$accum = accum * momentum + grad$$ $$var -= lr * accum$$ Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). accum: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). lr: A `Tensor`. Must have the same type as `var`. Learning rate. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. momentum: A `Tensor`. Must have the same type as `var`. Momentum. Must be a scalar. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. use_nesterov: An optional `bool`. Defaults to `False`. If `True`, the tensor passed to compute grad will be var - lr * momentum * accum, so in the end, the var you get is actually var - lr * momentum * accum. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Nsparse_apply_momentum op does not support eager execution. Arg 'out' is a ref.NFrrpSparseApplyMomentumrr$rr%)rrDrrrxrrrpr#r3r4r5r6r7r8r9r:s r;sparse_apply_momentumrs4F    0h..0$ #\\ g hhK""; >+L##L.A,'883e'.+6,8t E!QX QK' ""$3%%c*J  ,m  /  02F::L |VW> (' .r=zraw_ops.SparseApplyMomentumc td)Nrr?) rrDrrrxrrrpr#r@s r;$sparse_apply_momentum_eager_fallbackr8rr=TV_SparseApplyProximalAdagrad_T&TV_SparseApplyProximalAdagrad_Tindicesc tjxstj} | j} | jr t d|d}t j |d}tjd||||||||| \} } } } | dd}t jrYd| jdd| jdd| jdf}| j}t jd||||\}|S) aSparse update entries in '*var' and '*accum' according to FOBOS algorithm. That is for rows we have grad for, we update var and accum as follows: $$accum += grad * grad$$ $$prox_v = var$$ $$prox_v -= lr * grad * (1 / sqrt(accum))$$ $$var = sign(prox_v)/(1+lr*l2) * max{|prox_v|-lr*l1,0}$$ Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). accum: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). lr: A `Tensor`. Must have the same type as `var`. Learning rate. Must be a scalar. l1: A `Tensor`. Must have the same type as `var`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `var`. L2 regularization. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. use_locking: An optional `bool`. Defaults to `False`. If True, updating of the var and accum tensors will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Vsparse_apply_proximal_adagrad op does not support eager execution. Arg 'out' is a ref.NFrSparseApplyProximalAdagradrr$rr%)rrDrr[r\rrxrr#r3r4r5r6r7r8r9r:s r;sparse_apply_proximal_adagradr">s>    0h..0$ #\\ o ppK""; >+'88$#Urb)+$2=DJ!QX QK' ""$3%%c*J  ,m  /1F::L $lFGE (' .r=z"raw_ops.SparseApplyProximalAdagradc td)Nr r?) rrDrr[r\rrxrr#r@s r;,sparse_apply_proximal_adagrad_eager_fallbackr$wsmnnr='TV_SparseApplyProximalGradientDescent_T.TV_SparseApplyProximalGradientDescent_Tindicesc tjxstj}|j} | jr t d|d}t j |d}tjd|||||||| \} } } } | dd} t jrYd| jdd| jdd| jdf}| j}t jd||| | \} | S) aCSparse update '*var' as FOBOS algorithm with fixed learning rate. That is for rows we have grad for, we update var as follows: $$prox_v = var - alpha * grad$$ $$var = sign(prox_v)/(1+alpha*l2) * max{|prox_v|-alpha*l1,0}$$ Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). alpha: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. l1: A `Tensor`. Must have the same type as `var`. L1 regularization. Must be a scalar. l2: A `Tensor`. Must have the same type as `var`. L2 regularization. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var and accum. use_locking: An optional `bool`. Defaults to `False`. If True, the subtraction will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. _sparse_apply_proximal_gradient_descent op does not support eager execution. Arg 'out' is a ref.NFr"SparseApplyProximalGradientDescentrr$rr%)rrzr[r\rrxrr#r3r4r5r6r7r8r9r:s r;&sparse_apply_proximal_gradient_descentr*}s6    0h..0$ #\\ x yyK""; >+'88,#Ur13$6=:E37 9!QX QK' ""$3%%c*J  ,m  /1F::L ,lFGM (' .r=z*raw_ops.SparseApplyProximalGradientDescentc td)Nr(r?) rrzr[r\rrxrr#r@s r;5sparse_apply_proximal_gradient_descent_eager_fallbackr,svwwr=TV_SparseApplyRMSProp_TTV_SparseApplyRMSProp_Tindicesc tjxstj} | j} | jr t d| d} t j | d} tjd|||||||||| |  \} } }}|dd}t jrYd|jdd|jdd|jdf}|j}t jd||||\}|S) aUpdate '*var' according to the RMSProp algorithm. Note that in dense implementation of this algorithm, ms and mom will update even if the grad is zero, but in this sparse implementation, ms and mom will not update in iterations during which the grad is zero. mean_square = decay * mean_square + (1-decay) * gradient ** 2 Delta = learning_rate * gradient / sqrt(mean_square + epsilon) $$ms <- rho * ms_{t-1} + (1-rho) * grad * grad$$ $$mom <- momentum * mom_{t-1} + lr * grad / sqrt(ms + epsilon)$$ $$var <- var - mom$$ Args: var: A mutable `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `complex64`, `int64`, `qint8`, `quint8`, `qint32`, `bfloat16`, `qint16`, `quint16`, `uint16`, `complex128`, `half`, `uint32`, `uint64`. Should be from a Variable(). ms: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). mom: A mutable `Tensor`. Must have the same type as `var`. Should be from a Variable(). lr: A `Tensor`. Must have the same type as `var`. Scaling factor. Must be a scalar. rho: A `Tensor`. Must have the same type as `var`. Decay rate. Must be a scalar. momentum: A `Tensor`. Must have the same type as `var`. epsilon: A `Tensor`. Must have the same type as `var`. Ridge term. Must be a scalar. grad: A `Tensor`. Must have the same type as `var`. The gradient. indices: A `Tensor`. Must be one of the following types: `int32`, `int64`. A vector of indices into the first dimension of var, ms and mom. use_locking: An optional `bool`. Defaults to `False`. If `True`, updating of the var, ms, and mom tensors is protected by a lock; otherwise the behavior is undefined, but may exhibit less contention. name: A name for the operation (optional). Returns: A mutable `Tensor`. Has the same type as `var`. Nsparse_apply_rms_prop op does not support eager execution. Arg 'out' is a ref.NFrSparseApplyRMSProprr$rr%)rrrrrFrrrrxrr#r3r4r5r6r7r8r9r:s r;sparse_apply_rms_propr2s P    0h..0$ #\\ g hhK""; >+'88#"#"#'/t&-;#' )!QX QK' ""$3%%c*J  ,m  /1F::L lFG= (' .r=zraw_ops.SparseApplyRMSPropc td)Nr0r?) rrrrrFrrrrxrr#r@s r;$sparse_apply_rms_prop_eager_fallbackr4rr=)FN)FTN)FFN(#__doc__ collectionstensorflow.pythonrtensorflow.python.eagerrr&rrrr*tensorflow.python.frameworkrrtensorflow.security.fuzzing.pyr_atypesr _op_def_registryr rr r,"tensorflow.python.util.deprecationr tensorflow.python.utilr _dispatch tensorflow.python.util.tf_exportrtypingrrrtyping_extensionsrrboolr< to_raw_opr!rArCrKrIrMrOrUrSrWrXInt64rbr`rdrfrkrirmrnrursrwryrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrResourcerrrrrrrr rr rrrrrrrrrr r%r"r$r(r-r+r,r/r3r1r2r5r9r7r8r;r@r=r?rBrFrDrErIrMrKrLrOrTrQrRrWr[rYrZr]rar_r`rcrgrerfrjrnrlrmrprtrrrsrvrwr}rzr|rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr r r r rrrrrrrrrrrr"r!r$r%r&r*r)r,r-r.r2r1r4r=r;rHs6 67179FK3IC86%%'-/ACWYlnBSUcetvEGVXfhxzJL[]np@BRTdfvxGH6y&6!676IcK[F[<\6ajknqAlAbB6QZ[^`p[pQq6w@ADFVAVwW6`ijmoj`@6IRSVXhShIi6t}~ACS~StT6\efik{f{\|6JN6dmnqsCnCdD6p/i-.~t~~m/LM _i5E0E&F_9UXZjUjKk_pyz}@P{PqQ_`ijmoj`@_FOPSUePeFf_oxy|~NyNoO_XabegwbwXx_CLMPRbMbCc_ktuxzJuJkK_Z^_nwx{}MxMnN_13EG[]psDFWYgixzIKZ\jl|~NP_artDFVXhjz|KL2 #'9"9:29SRdMdCe2u~@CEW@WvX2^ghkmh^@2GPQTVhQhGi2t}~ACU~UtV2^ghkmh^@2NR2hqruwIrIhJ2h3 12>4>>.3QR `y6H1H'I`R[\_as\sRt`ENORTfOfEg`mvwz|NwNmO`V_`cew`wVx`CLMPRdMdCe`mvwz|NwNmO`^b`r{|AS|SrT`/1CEY[nqBDUWegvxGIXZhjz|LN]_prBDTVfhxzIJ0y&7!7803PaKaAb0hqruxIsIiJ0R[\_ar\rRs0AE0Z^0s|}@BS}SsT0d1y/0 1NO _i5F0F&G_PYZ]_pZpPq_xABEGXBXxY_ajknpAkAaB_QU_ei_yBCFHYCYyZ_35GI]_ruFHY[ikz|KM\^ln~@PRactvFHXZjl|~MN1)C)<$<=1U^_bdw_wUx1Ybcfh{c{Y|1DMNQSfNfDg1mvwz|OwOmP1V_`cex`xVy1HILNaIab1qz{~@G@M@M{MqN1\`1v@CEX@XvY1f534^T^^DT5UVb38K3K)LbdmnqtGoGeHbhqruwJrJhKbS\]`bu]uSvb|EFIK^F^|_benortGoGeHbNWX[]pXpNqb@IJMOVO\O\J\@]blpb@IJMObJb@cb35GI]_ruFHY[ikz|KM\^ln~@PRactvFHXZjl|~MN2)C)<$<=2iPSUhPhFi2oxy|RzRpS2^ghkm@h@^A2IRSVXkSkIl2z~2SW2luvy{NvNlO2h534^T^^DT5UVb38K3K)LbU^_bdw_wUxbHILNaIabbmvwz|OwOmPbXabegzbzX{bJNb^bbr{|AT|TrUb)+=?SUhj{~OQ_aprACRTbdtvFHWYjl|~NP`brtCD@Ic>12@ynAT7U@ZcdgiwdwZx@HQRUWeReHf@u~BDRRuS@YbcfhvcvYw@@IJMO]J]@^@gpqtvDqDgE@PYZ]_mZmPn@v@CES@SvT@bf@{@U^_bdr_rUs@D +I) *>4>>*+E F \9S.-@#A\iPSUcPcFd\irsvyGtGjH\W`adftatWu\DMNQSaNaDb\hqruwErEhF\OXY\^lYlOm\v@CES@SvT\_hiln|i|_}\ENORTbObEc\rv\FJ\ZcdgiwdwZx\/1CEY[nqBDUWegvxGIXZhjz|LN]_prBDTVfhxzIJ0 #'8"890iM^H^>_0enoruFpFfG0PYZ]_pZpPq0HILN_I_`0hqruwHrHhI0QZ[^`q[qQr0@D0ZcdgizdzZ{0d1y/01OP `y6G1G'H`YWZ\mWmMn`t}BDUUuV`_hilni_@`NWX[]nXnNo`w@ADFWAWwX``ijmo@j@`A`PT`dmnqsDnDdE`$$?ASUik~ARTeguwFHWYhjxzJL\^mo@BRTdfvxHJYZE30I+I!JEPYZ]_xZxPyE@IJMOhJh@iEpyz}XzXpYE_hilnGiG_HEOXY\^wYwOxEDMNQSlNlDmExABEG`B`xaEirsvxQsQiRE`dEzCDGIbDbzcENAy!?@PgAhii #?X:X0Yi_hiloHjH`IiOXY\^wYwOxiHILNgIghinwx{}VxVnWi^ghkmFhF^GiS\]`b{]{S|iGPQTVoQoGpixABEG`B`xaiptiDMNQSlNlDmi)+=?SUhj{~OQ_aprACRTbdtvFHWYjl|~NP`brtCD;Ic>12;9S.EX;Y;clmpsAnAdB;JSTWYgTgJh;nwx{}KxKnL;R[\_ao\oRp;v@CES@SvT;`ijmo}j}`~;LP;nr;HQRUWeReHf;z +I) *>4>>*+E F \9S.-@#A\)TWYgTgJh\r{|BP}PsQ\YbcfhvcvYw\}FGJLZGZ}[\ajknp~k~a\ENORTbObEc\oxy|~LyLoM\\`\y}\MVWZ\jWjMk\-/ACWYlnBSUcetvEGVXfhxzJL[]np@BRTdfvxGH?y&6!67? #O_J_@`?jstwzJuJkK?S\]`br]rSs?yBCFHXCXyY?_hiln~i~_?ENORTdOdEe?u~BDTTuU?ajknp@k@aA?OS?qu?KTUXZjUjKk?B/i-.~t~~m/LM _i5E0E&F_yY\^nYnOo_zCDGIYDYzZ_bkloqAlAbB_HQRUWgRgHh_nwx{}MxMnN_T]^acs^sTt_DMNQScNcDd_pyz}OzOpP__c_|@_PYZ]_oZoPp_$$?ASUik~ARTeguwFHWYhjxzJL\^mo@BRTdfvxHJYZ$ #/H*H I$R[\_az\zR{$ENORTmOmEn$|@$V_`ce~`~V$LAy!?@PfAghhy>W9W/XhajknqJlJbKhT]^ac|^|T}hLPh`ijmoHjH`Ih13EG[]psDFWYgixzIKZ\jl|~NP_artDFVXhjz|KL7 #'9"9:79SRdMdCe7ktux{MvMlN7V_`cew`wVx7DMNQSeNeDf7tx7MQ7gpqtvHqHgI7r3 12>4>>.3QR `y6H1H'I`R[\_as\sRt`{DEHJ\E\{]`enortFoFeG`S\]`bt]tSu`DH`X\`luvy{MvMlN`35GI]_ruFHY[ikz|KM\^ln~@PRactvFHXZjl|~MN0)C)<$<=0)CQdLdBe0ktux{NvNlO0Zcdgi|d|Z}0KTUXZmUmKn0v@CEX@XvY0ajknpCkCaD0RV0luvy{NvNlO0d534^T^^DT5UVb38K3K)LbQZ[^`s[sQtb{DEHJ]E]{^birsvxKsKiLbZcdgi|d|Z}bENORTgOgEhbpyz}RzRpSbbfbv@CEX@XvYb$$?ASUik~ARTeguwFHWYhjxzJL\^mo@BRTdfvxHJYZ. #/H*H I.R[\_az\zR{.BKLOQjLjBk.qz{~@Y{YqZ.`ijmoHjH`I.QZ[^`y[yQz.HL.bkloqJlJbK.`Ay!?@PfAghhy>W9W/XhajknqJlJbKhQZ[^`y[yQzh@IJMOhJh@ihoxy|~WyWoXh`ijmoHjH`IhX\hluvy{TvTlUh%,,OQcey|OQbduwEGVXgixzHJZ\ln}PRbdtvFHXZi%j!,38Y3Y)Z,clmpsTnTdU,[dehjKeK[L,R[\_aB\BRC,LUVY[|V|L},KO,enortUoUeV,\ Qy)OPQ_QUQ_Q_`RA Bq #GhBh8iqr{|Bc}csdqjstwyZtZj[qajknpQkQaRq[dehjKeK[Lq[_qoxy|~_y_o`q/1CEY[nqBDUWegvxGIXZhjz|LN]_prBDTVfhxzIJ: #'8"89:yN_I_?`:gpqtwHrHhI:OXY\^oYoOp:w@ADFWAWwX:dmnqsDnDdE:PYZ]_pZpPq:yBCFHYCYyZ:hl:BKLOQbLbBc:x1y/01OP `y6G1G'H`iX[]nXnNo`vADFWAWwX`^ghkm~h~^`FOPSUfPfFg`s|}@BS}SsT`_hilni_@`HQRUWhRhHi`x|`LUVY[lVlLm`##=?QSgi|PRcesuDFUWfhvxHJZ\km~@PRbdtvFHWX: #w/?/?*? @: YsT[TdTdOdEe: jstwzAzJzJuJkK: ZcdgiAdAZB: HQRUWoRoHp: yBCFH`C`ya: jstwyQtQjR: ]fgjlDgD]E: MVWZ\tWtMu: CG: v?i =>~t~~Nd?efyg>N>N9N/OT]^acjcscs^sTtzCDGIPIYIYDYzZirsvxPsPiQW`adf~a~WHQRUWoRoHpyBCFH`C`yaluvy{SvSlT\efikCfC\DSW"%%ACUWknACTVgiwyHJY[jlz|LN^`oqBDTVfhxzJL[\6 30@0@+@!A6 )TWY`YiYiTiJj6 {DEHJQJZJZEZ{[6 ajknpJkJaK6 R[\_a{\{R|6 GPQTVpQpGq6 yBCFHbCbyc6 qu6 nC "AB>4>>RiCjk #w?O?O:O0PYbcfhohxhxcxYyJSTWY`YiYiTiJjpyz}YzYpZajknpJkJaKV_`ce`V@HQRUWqRqHrAE"$$?ASUik~ARTeguwFHWYhjxzJL\^mo@BRTdfvxHJYZ3 #w/?/?*? @3 SVX_XhXhShIi3 oxy|XzXpY3 ajknpIkIaJ3 X\3 qu3 hAy!?@PfAghyg>N>N9N/OXabegngwgwbwXxHILNgIghpyz}XzXpYhl|@(&&CEWYmpCEVXiky{JL[]ln|~NP`bqsDFVXhjz|LN]^9 9S'2B2B-B#C9 [dehjqjzjzez[{9 \efikrk{k{f{\|9 DMNQSnNnDo9 u~BD__u`9 fopsuPpPfQ9 W`adfAaAWB9 QZ[^`g`m`m[mQn9 |@9 tE#CD^T^^TmEno)CAQAQ4>>J];^_Ic7;K;K6K,LQZ[^`g`p`p[pQqvADFMFVFVAVwWfopsuKpKfL[dehj@e@[AGPQTVlQlGmv@CE[@[v\enortJoJeKV_`ce{`{V|DMNQSiNiDjy}MQ*%,,OQcey|OQbduwEGVXgixzHJZ\ln}PRbdtvFHXZi%j!B )C9I9I4I*JB yY\^e^n^nYnOoB t}BDKDTDTTuUB ]fgjlsl|l|g|]}B LUVY[|V|L}B LUVY[|V|L}B CLMPRsMsCtB }FGJLmGm}nB w@ADFgAgwhB s|}@Bc}csdB luvy{\ v\ l] B k o B F Qy)OPQ_QUQ_Q_aARB C3HXHXCX9Y^ghkmtm}m}h}^~DMNQSZScScNcDdluvy{B{K{KvKlL[dehjKeK[L[dehjKeK[LR[\_aB\BRCLUVY[|V|L}FOPSUvPvFwBKLOQrLrBs{D E H J k E k {l {  &$$?ASUik~ARTeguwFHWYhjxzJL\^mo@BRTdfvxHJYZ3 30@0@+@!A3 iPSU\UeUePeFf3 luvy|UwUmV3 _hilnGiG_H3 V_`ce~`~V3 GPQTVoQoGp3 xABEG`B`xa3 os3 hAy!?@PgAhi  #w?O?O:O0P U^_bdkdtdt_tUu |EFIKdFd|e nwx{}VxVnW enortMoMeN V_`ce~`~V GPQTVoQoGp C  %,,OQcey|OQbduwEGVXgixzHJZ\ln}PRbdtvFHXZi%j!H )C9I9I4I*JH PYZ]_f_o_oZoPpH vADFMFVFVAVwWH ^ghkmtm}m}h}^~H DMNQStNtDuH |EFIKlFl|mH yBCFHiCiyjH u~BDeeufH nwx{}^x^n_H mqH R Qy)OPQ_QUQ_Q_aARB C3HXHXCX9Y_hilnun~n~i~_FOPSU\UeUePeFfmvwz|C|L|LwLmMS\]`bC]CSDKTUXZ{U{K|HQRUWxRxHyDMNQStNtDu}FGJLmGm}n}A&!!9;MOcex{LN_aoq@BQSbdrtDFVXgiz|LN^`prBDSTB YsG,<,<'<=B iPSU\UeUePeFfB pyz}@G@P@P{PqQB Ybcfh~c~YB ENORTjOjEkB qz{~@V{VqWB ]fgjlBgB]CB OXY\^tYtOuB CGB eiB F;I9:>4>>J];^_Ic7;K;K6K,LU^_bdkdtdt_tUu@IJMOVO_O_J_@`hqruwMrMhNT]^acy^yTz@IJMOeJe@fluvy{QvQlR^ghkmChC^DSWpt*##=?QSgi|PRcesuDFUWfhvxHJZ\km~@PRbdtvFHWXE #w/?/?*? @E SVX_XhXhShIiE s|~ACJCSCS~StTE \efikCfC\DE JSTWYqTqJrE xABEG_B_x`E fopsuMpMfNE ^ghkmEhE^FE R[\_ay\yRzE HLE jnE L?i =>~t~~Nd?efyg>N>N9N/OXabegngwgwbwXxCLMPRYRbRbMbCcktuxzRuRkSYbcfh@c@YAGPQTVnQnGou~BD\\u]mvwz|TwTmUajknpHkHaIX\uy*%,,OQcey|OQbduwEGVXgixzHJZ\ln}PRbdtvFHXZi%j!( 38H8H3H)I( R[\_bC]CSD( MVWZ\}W}M~( LP( R Qy)OPQ_QUQ_Q_`RA B  #wGWGWBW8X ajknqRlRbS \efikLfL\M \`  #**KM_auxKM^`qsACRTcetvDFVXhjy{LN^`prBDTVe#f; yg6F6F1F'G; PYZ]_f_o_oZoPp; vADFeAewf; nwx{}\x\n]; irsvxWsWiX; fj; C; xMY'KL^T^^\yMz{iWEUEU@U6V_hilnun~n~i~_FOPSUtPtFu}FGJLkGk}lxABEGfBfxgvzJN*%%ACUWknACTVgiwyHJY[jlz|LN^`oqBDTVfhxzJL[\: 30@0@+@!A: )TWY`YiYiTiJj: pyz}@Z{Zq[: clmprLmLcM: YbcfhBcBYC: QU: jn: vC "AB>4>>RiCjk #w?O?O:O0PYbcfhohxhxcxYy@IJMOiJi@jr{|A[|[r\hqruwQrQhRaeuy(&&CEWYmpCEVXiky{JL[]ln|~NP`bqsDFVXhjz|LN]^3 9S'2B2B-B#C3 RUW^WgWgRgHh3 nwx{~YyYoZ3 enortOoOeP3 ^ghkmHhH^I3 QZ[^`{[{Q|3 DMNQSnNnDo3 }A3 hE#CD^T^^TmEno)CAQAQ4>>ZwKxyiWEUEU@U6V_hilnun~n~i~_JSTWY`YiYiTiJjr{|A_|_r`ktuxz_u_k`fopsuSpSfTZcdgiGdGZHNWX[]{X{N|LUVY[yVyLzFOPSUsPsFtC G ` d .)00WYknBDWYjl}MO^`oq@BPRbdtvEGXZjl|~NP`bq)r%/67egvyH0I,B iW=M=M8M.NB W`adfmfvfvavWwB ~GHKMrHr~sB {DEHJoEo{pB {DEHJvEv{wB CLMPRwMwCxB FJB _cB F$Y9-W#XYgY]YgYgiMZN$O YsGL\L\G\=]fopsu|vFvFqFgGMVWZ\AWAMBJSTWY~T~JJSTWYETEJFR[\_aF\FRGVZjn,$++MOacwzMO`bsuCETVegvxFHXZjl{}NP`brtDFVXg$h *12[]ln}*~'@ #w7G7G2G(H@ QZ[^`g`p`p[pQq@ xABEGgBgxh@ pyz}_z_p`@ ktuxzauakb@ nwx{}]x]n^@ lp@ EI@ BOi(MN~t~~^|O}~ygFVFVAV7W`ijmovoojaAGPQTVvQvGwHILNnInozCDGIpDpzq}FGJLlGl}m|@PT,+22[]orFH[]npACQSbdsuDFTVfhxzIK\^np@BRTdfu+v'189ikz}L2M.9 #w?O?O:O0P9 YbcfhohxhxcxYy9 @IJMOvJv@w9 }FGJLsGs}t9 zCDGIpDpzq9 yBCFHoCoyp9 {DEHJxEx{y9 GK9 t&]Y/[%\]k]a]k]kmS^T&U"ygN^N^I^?_hqruw~xHxHsHiIOXY\^EYEOFLUVY[BVBLCIRSVXSI@HQRUW~R~HJSTWYGTGJHW[&3::kmBVXkm~@QSacrtCETVdfvxHJY[ln~@PRbdtvE3F/9@Ay|KM\:]66 3HXHXCX9Y6 bkloramacb6 hqruwfrfhg6 mvwz|kwkml6 t}~ACr~rts6 ~GHKMCHC~D6 RV6 n.mY7k-lm{mqm{m{}lnm.n* RUW^WgWgRgHhqz{~Ap|prqw@ADFuAuwv|EFIKzFz|{CLMPRAMACBMVWZ\RWRMSbf$#**KM_auxKM^`qsACRTcetvDFVXhjy{LN^`prBDTVe#f)01Y[jl{)|&@ #w7G7G2G(H@ iX[]d]m]mXmNn@ u~@CELEUEU@UvV@ \efikJfJ\K@ R[\_a@\@RA@ MVWZ\{W{M|@ GPQTVuQuGv@ ~GHKMlHl~m@ xABEGmBmxn@ |@@ BMY'KL^T^^\zM{|ygFVFVAV7W]fgjlsl|l|g|]}ENORT[TdTdOdEektuxzYuYkZajknpOkOaP\efikJfJ\KV_`ceD`DVEMVWZ\{W{M|GPQTV|Q|G}LP(##=?QSgi|PRcesuDFUWfhvxHJZ\km~@PRbdtvFHWX")*K_^m"n0y.F)FG0PYZ]_wZwPx0IRSVXpSpIq0w@ADF^A^w_0fopsuMpMfN0Ybcfh@c@YA0IRSVXpSpIq0|EFIKjFj|k0y}0S\]`bz]zS{0d?i =>~t~~Nc?degi=U8U.Vg_hiloGjG`HgXabegbX@gFOPSUmPmFngu~BD\\u]ghqruwOrOhPgXabegbX@gKTUXZyUyKzgIMg]fgjlDgD]Eg"";=OQegz}NPacqsBDSUdftvFHXZik|~NP`brtDFUV!()I?\k!l4i-D(DE4iX[]tXtNu4|EFIKbFb|c4ktuxzQuQkR4]fgjlJgJ]K4Y]4rv4KTUXZqUqKr4l=Y;<^T^^L`=abfYs%>?A RUWkRkHlAvADFZAZw[AclmprFmFcGAR[\_a|\|R}ACLMPRfMfCgAmvwz|PwPmQAW`adfzazW{AGPQTVjQjGkAy}A[_Au~BDXXuYAF7)56~t~~FW7XYc)C9M4M*NcW`adfzazW{cFOPSUiPiFjcr{|AU|UrVcajknpKkKaLcR[\_au\uRvc|EFIK_F_|`cfopsuIpIfJcV_`cey`yVzcIMcfjczCDGI]D]z^c!!9;MOcex{LN_aoq@BQSbdrtDFVXgiz|LN^`prBDST '(GZi jEi-C(CDEYWZ\rWrMsE~GHKMcHc~dEluvy{QvQlRE]fgjlIgI]JEPYZ]_uZuPvE|EFIKaFa|bEhqruwMrMhNE^ghkmChC^DEPYZ]_uZuPvEDHEfjE@ I J M O e J e @ f EN;I9:>4>>J^;_`fYs~t~~Nc?degi=U8U.Vg_hiloGjG`HgNWX[]uXuNvg~GHKMeHe~fgqz{~@_{_q`gluvy{SvSlTgcggw{gKTUXZrUrKsg#**KM_auxKM^`qsACRTcetvDFVXhjy{LN^`prBDTVe#f)01Y[jl{)|&4y6U1U'V4_hiloNjN`O4U^_bdC_CUD4JSTWYxTxJy4HILNmImn4v@CEd@dve4pyz}ezepf4tx4NWX[]|X|N}4lMY'KL^T^^\yMz{oiEd@d6eonwx{~]y]o^odmnqsRnRdSoYbcfhGcGYHoNWX[]|X|N}oENORTsOsEtoHILNtItuoDHoXabegFbFXGo+22[]orFH[]npACQSbdsuDFTVfhxzIK\^np@BRTdfu+v'189ikz}L2M.2 #?f:f0g2pyz}@g{gqh2nwx{}dxdne2ktuxzauakb2jstwy`t`ja2luvy{ivilj2x|2R[\_aH\HRI2h&]Y/[%\]k]a]k]kmS^T&U"xyNuIu?vx@IJMOvJv@wx}FGJLsGs}txzCDGIpDpzqxyBCFHoCoypx{DEHJxEx{yxHLx\efikRfR\Sx"";=OQegz}NPacqsBDSUdftvFHXZik|~NP`brtDFUV!()I?\k!l>y.E)EF>IVY[rVrLs>{DEHJaEa{b>hqruwNrNhO>V_`ce|`|V}>IRSVXoSoIp>{DEHJaEa{b>jstwyPtPjQ>\efikIfI\J>X\>r{|AX|XrY>@=Y;<^T^^La=bcgi=T8T.Ug[dehkBfB\CgJSTWYpTpJqgw@ADF]A]w^genortKoKeLgXabeg~b~XgJSTWYpTpJqgyBCFH_C_y`gktuxzXuXkYghlg|EFIKbFb|cgr=