Handbook
Glossary
gelu ( x -- x' )
Vocabulary
machine-learning
.
functions
Inputs
x
an
object
Outputs
x'
an
object
Definition
USING:
kernel
math
math.constants
math.functions
math.vectors
sequences
;
IN:
machine-learning.functions
:
gelu
( x -- x' )
dup
dup
3
v^n
0.044715
v*n
v+
2
pi
/
sqrt
v*n
[
tanh
]
map
1
v+n
v*
0.5
v*n
;